Type</th>	Usage</th>	Guarantees</th></tr></thead>
Affine types</strong></td>	at most once</td>	No more “use after free”</td></tr>
Linear types</strong></td>	exactly once</td>	No more memory leaks</td></tr>
Ordered types</strong></td>	exactly once, in-order</td>	Stable memory locations</td></tr> </tbody></table> I won’t bore you with terms like “contraction” and “weakening” here. You should read the wikipedia page</a> if you want to learn more. But specifically for Rust: this is why we’ve been working on new traits like `Move</code></a> and` `Forget</code></a>:</p>` !Forget</code> unlocks linear types</li> !Move</code> unlocks ordered types 1</a></sup></li> </ul> ^1</sup>Move unlocks ordered types in a way that actually composes</a> with the rest of the language, unlike Pin</code>. To be fully ergonomic does however also depend on emplacement in the language</a>, which I believe is best expressed as an effect.</p> </div> Refinement types</h2> “Use after free” is more formally known as a temporal memory safety violation</em>. Its sibling is the “out of bounds error” which is also known as a spatial memory safety violation</em> (ref</a>). In Rust the borrow checker can entirely statically guarantee we never run into use after free bugs. However if we want to relax those rules a little, we can opt-in to using types like Rc</code> and Arc</code> which check those properties at runtime instead.</p> When it comes to out of bounds checks things are however a little different. The default is typically to check bounds at runtime, which may sometimes be omitted by the compiler - but only as an optimization. That means we’re trading away runtime performance for memory safety.</p> But what if we could trade away compile time for memory safety instead? Type systems which can attach additional guarantees to existing types are called refinement type systems</em>. And for Rust we’re experimenting with an ultra-lightweight version of this we’re calling pattern types</em>.</p> Pattern types use Rust’s pattern syntax to annotate existing types. Take for example NonZeroUsize</code>: this has historically been backed by a ton of custom compiler optimizations to enable the use of niches in layout. But with pattern types we can get those same optimizations automatically by refinining</em> the usize</code> type with a pattern:</p> type </span>NonZeroUsize = </span>usize</span> is </span>1</span>..; </span>// ^^^^^^ refinement using patterns </span></code></pre> A closely related feature to pattern types are view types</em>, which would enable the compiler to consider disjoint patterns when reasoning about aliasing (ref</a>). These are great because they would allow holding two mutable references to the same type, as long as each reference doesn’t look at any of the other fields.</p> I see pattern types and view types as the way we can make Rust’s borrow checker story even better</em>. Both by removing the fundamental tradeoff between runtime checks vs memory safety (pattern types). As well as making even more valid borrows expressible (view types).</p> Conclusion</h2> There are a lot of things in Rust to be excited about. I love the work that’s happening in improving the formalisms in the language, improvements in the compiler, and in the ecosystem. And there are many more language improvements in the works which I haven’t mentioned but I’m excited for (I’m looking at you, reflection).</p> But me personally? I want for Rust to become the safest darn production-grade language in existence. Because though we’re not doing too bad, we’re no Ada/SPARK yet. But also: working on features like these is what I find interesting and exciting. It’s why I volunteer to work on Rust.</p> And while figuring out how to fundamentally improve Rust isn’t easy or quick. I believe it is definitely possible, eventually worthwhile, and in my opinion: a lot of fun!</p> Syntactic musings on the fallibility effect Wed, 17 Dec 2025 00:00:00 +0000 I believe it must have been about three or four years ago when Rust added the unstable yeet</code> keyword on nightly which can be used to return new errors in try</code>-functions. More recently Rust has added the unstable bikeshed</code> keyword on nightly which gives try</code>-blocks and closures the ability to express which kind of error they operate on. We’ve been deferring making decisions of error handling syntax for a while now, and I think that’s actually quite reasonable.</p> But if we want things to head for stable, we’re eventually going to want to decide on syntax. And so I think it’s not a bad idea to start working through the entire syntactic space here. And because things are unstable I guess it’s unavoidable to also work through the semantic space here. I guess this is a long-winded way of saying: in this post I’m sharing my opinions on Rust’s effect for fallibility. And I’m doing that mostly because I think it’s fun to think about.</p> Defining the problem space</h2> For Rust we can categorize the error handling space into three categories. The first category are the places which carry the effect type notation, of which there are three. These are items in the language where for which we want to say: “Hey, this right here is operating with an error handling context”</em>. Those items are:</p> blocks ({}</code>)</li> functions (fn f() {}</code>)</li> closures (\|\| {}</code>)</li> </ul> The second category are the effect operators. These are used inside of the items marked with the fallibility effect, and allow us to operate on them. There are three kinds of operations we need to support:</p> create (e.g. yield</code> for iteration)</li> propagate (e.g. .await</code> for async)</li> consume (e.g. block_on</code> for async, for..in</code> for iteration)</li> </ul> Both of the categories we’ve seen so far apply to all of Rust’s control-flow effects (async, fallibility, and iteration). But the third category is unique to fallibility only, and that is: specialization. Fallibility has two modes it can operate in:</p> abstract, where the effect is reified into an abstract -> impl Try</code></li> concrete, where the effect is reified into a specific type (e.g. -> io::Result</code>)</li> </ul> You can almost think of fallibility as being two separate effects in a trenchcoat. Or maybe as “concrete” being a specialized version of “abstract”. But since both flavors of fallibility are so similar in both purpose and usage, to an end-user we will both to feel similar to one another.</p> Fallible functions</h2> So for the effect type we have to account for functions, closures, and blocks. For both of these we need to be able to support both abstract and concrete variants. Or put differently: we need to be able to optionally be able spell out both the explicit type when needed.</p> Starting with functions. Swift uses the throws</code> keyword to annotate a function as fallible. I believe this matches Java, Kotlin, and Scala. This keyword can optionally carry a concrete type, and I think that’s perfect. Here is how I would adapt that for Rust:</p> fn </span>foo</span>() {} </span>// `-> ()` (base) </span>fn </span>foo</span>() throws {} </span>// `-> impl Try` </span>fn </span>foo</span>() throws Err(io::Error) {} </span>// `-> io::Result<()>` </span>fn </span>foo</span>() throws None {} </span>// `-> Option<()>` </span>fn </span>foo</span>() throws None -> </span>i32 </span>{} </span>// `-> Option<i32>` </span></code></pre> The notation here borrows from pattern types. None</code> and Err</code> imported as part of the Rust prelude, and just like we can write None</code> and Err</code> are in function bodies, we should be able to use them freely in signatures as well. And since we know they are part of an impl Try</code>, we know what the actual type is. The only open question I have is express that in the impl Try</code> trait itself - but the Try</code> trait is still unstable, so that’s something we can figure out.</p> In terms of location this follows what Swift does, listing effects after the argument list but before the arrow. This to me makes sense because effects describe properties of the function itself. And because throws</code> says something about the function’s outputs, I expect it to be listed after</em> the argument list. And because we don’t want it to be confused with the logical return type, it makes sense for it to be listed before</em> the return type.</p> Something to be aware of also is that it’s notationally important that we are always able strip the effect from the function signature and still end up with a valid function signature. That’s an important property we need to be able to group effects together and reason about them in sets or as variables. Those include things like: effect aliases, associated effects, and effect generics:</p> // Adding or removing effects from functions should </span>// not affect the rest of the function signature: </span>async </span>fn </span>foo</span>() -> </span>i32</span>; </span>// async effect </span>const fn </span>foo</span>() -> </span>i32</span>; </span>// const effect </span>fn </span>foo</span>() throws None -> </span>i32</span>; </span>// fallible effect </span>fn </span>foo</span>() -> </span>i32</span>; </span>// no effects (baseline) </span> </span>// That keeps the door open to enable higher-order reasoning </span>// about effects. Using a Flix-inspired notation for fun: </span>effect ef = async + </span>const </span>+ throws None; </span>fn </span>foo</span>() -> </span>i32 </span>\ ef; </span></code></pre> Maybe there’s something to be said for going all the way with our notation and start the migration to a general-purpose effect notation. But I’m somewhat intentionally trying to scope things down here and argue for more localized decisions on how we notate fallibility. But I can be convinced to do something more generic/consistent if people feel strongly we should do it all in one go.</p> Fallible blocks and closures</h2> Blocks and closures are very similar in how they’re written; with the exception that closures must list their arguments and may list their return type. I think our block notation is a little goofy overall, and wish it would be able to list return types too. In my opinion Swift does this better in their do</code> statements, and like I kind of wish we would have that for blocks. But we don’t do that right now, so I think we should start by spelling closures and blocks as follows:</p> // abstract definitions </span>throws {} </span>// block notation </span>throws \|\| {} </span>// closure notation </span>throws \|\| -> </span>i32 </span>{} </span>// explicit return type </span>async throws \|\| {} </span>// mixing effects </span> </span>// concrete definitions </span>throws None {} </span>// block notation </span>throws None \|\| {} </span>// closure notation </span>throws None \|\| -> </span>i32 </span>{} </span>// explicit return type </span>async throws None \|\| {} </span>// mixing effects </span></code></pre> Personally I don’t love that the effects are listed here before</em> the argument list, while in fn() {}</code>-functions the effects are listed after</em> the argument list (and before the arrow). And so it’s weird to me if closures don’t follow this same syntactic pattern. But I think making that more consistent is something we can probably address later.</p> What probably stands out here though is that I’m opting to fully omit the try</code> keyword. The way I see it there are three options for how can we spell things here:</p> try {}</code>/try throws None {}</code></li> try throws {}</code>/try throws None {}</code></li> throws {}</code>/throws None {}</code></li> </ul> Personally I prefer the last one, omitting the try</code> keyword entirely. Though I like try</code> as a verb, it doesn’t carry any additional information that throws</code> doesn’t already. And so I remind myself of Stroustrup’s Rule</a> and away try</code> goes.</p> Operating on fallibility</h2> There are three kinds of operations we need to support for fallibility: create, propagate, and consume. The latter of the two we already have in the language, so let’s start there. To “forward” the fallibility effect in code, you can use what is officially called “the try propagation expression”</a> - though it is more commonly known as “the question mark operator”:</p> let</span> x = </span>foo</span>()?; </span>// `?` propagates the fallibility effect </span></code></pre> Try propagation expressions can only be used in functions which themselves return an impl Try</code> (either abstract or concrete). If we have a function which itself is infallible, or has a different Try</code> type, we will want to consume</em> the fallible effect. To do that we can use the match</code> keyword. If we are working with an abstract impl Try</code> implementation we will first need to cast it to a concrete type. But otherwise the logic is the same for both:</p> // `match` consumes a concrete `Result` type </span>match </span>foo</span>() { </span> Ok(x) => { .. } </span> Err(err) => { .. } </span>} </span> </span>// `match` consumes an abstract `impl Try` type </span>// NOTE: this should probably become a combinator on `Try` </span>let</span> res = </span>match </span>foo</span>().</span>branch</span>() { </span> Break(b) => FromResidual::from_residual(b), </span> Continue(c) => Try::from_output(c), </span>}; </span>match</span> res { </span> Ok(x) => { .. } </span> Err(err) => { .. } </span>} </span></code></pre> Then finally we have the missing “create” operator which allows you to create a new impl Try</code> type. On unstable today this is called yeet</code>, and though I think the name is funny, I think it should probably be called throw</code>. Here is how you would use it inside of functions:</p> throw None; </span>// -> `return None` </span>throw Err("</span>hello</span>") </span>// -> `return Err("hello".into())` </span></code></pre> And that is all three control-flow operations accounted for: ?</code>, match</code>, and throw</code>.</p> Notes on effect naming</h2> Though I like the throw</code> terminology overall, I think it could do with some tidying up. If we’re going to go with throw</code>, I want us to really</em> lean into it. To me using try</code> as a noun feels really unintuitive, and so I’d much rather we use nouns and verbs based on the throw</code> operation instead:</p> Try</code> → Throwable</code></li> try</code>/try..bikeshed</code> → throws</code></li> yeet</code> → throw</code></li> ?</code> (try propagation operator) → ?</code> (rethrow operator)</li> </ul> To me this degree of regularity is incredibly appealing. Aside from the general-purpose match</code> operation, all of these items seem clearly related to each other. And I think that’s really valuable! In a previous post I made a similar case for the iteration effect as well. Here is what I would change if I could (though I’m not saying we should):</p> IntoIterator</code> → Iterate</code></li> FromIterator</code> → Collect</code></li> </ul> Iterate</em> is the name of the operation, which itself returns an iterator</em> which can be iterated over. The async effect could probably also do with some normalization, since we’re currently mixing the terms “async”, “future”, and “poll” in ways that take time to internalize. I think some normalization of terminology here could be nice too (I’m musing, not prescribing):</p> IntoFuture</code> → Wait</code></li> Future</code> → Waitable</code></li> Future::poll</code> → Waitable::wait</code></li> Poll</code> → WaitableState</code></li> </ul> I guess though if we had a “throwable” and a “waitable”, then we might also want an “iterable”. I do like “iterable” less than “iterator”, but personally I like consistency even more. Though in the grand scheme of things: this specific name doesn’t matter too much either.</p> Motivating reification into an abstract type</h2> I didn’t really get into it before in this post, but I think we’re currently underestimating just how valuable it is for functions to be able to choose whether they are able to return an abstract impl Try</code> or a concrete Try</code> type. In my 2019 error handling survey</a> I highlighted the auto-enums crate which is able to automatically enable heterogenous types to be returned from function bodies.</p> I see this combination of features becoming the language-native way to provide 80% of the value of the anyhow::Error</code> type. Instead of casting error types to a Box<dyn Error + Send + Sync + 'static></code>, this would cast the error to an anonymous enum Error {}</code> with a variant per-type. That means:</p> no allocations are involved</li> each error variant is explicitly tracked</li> syntactically easy to write</li> natural upgrade path to explicit error types (e.g. thiserror</a>)</li> </ul> Here is an example of how I see that playing out. Function bar</code> here can either throw i32</code> or throw &'static str</code>. The idea is that the compiler would recognize this, create an anonymous enum that can hold both types, and then return that as the error type:</p> fn </span>foo</span>(</span>x</span>: </span>bool</span>) throws { </span>// `throws i32 \| &'static str` </span> </span>bar</span>(x)?; </span>} </span> </span>fn </span>bar</span>(</span>x</span>: </span>bool</span>) throws { </span> </span>if</span> x { </span> throw </span>12 </span>// `: i32` </span> } </span>else </span>{ </span> throw "</span>oops</span>" </span>// `: &'static str` </span> } </span>} </span></code></pre> The exact rules of how this would work would be a little subtle. We likely wouldn’t be able to just to coerce errors using .into()</code> the way we do today. But I think in the short term, we could get away with some basic rules to hold space for this in the future. So bare fn() throws {}</code> functions:</p> Should return an abstract impl Try</code> type</li> Should not call .into</code> as part of the ?</code> desugaring to coerce types</li> </ul> And I believe that’s about it. But I’m keen to hear from Scott (T-Lang) to tell me exactly what I’ve missed and why this might not be enough.</p> Closing words</h2> And that’s a complete design for a fallibility effect notation done. I know not everyone will like throw</code> as much as I do, but my hope is to steer any discussion here at least to some regularity. Say if we want to go with yeet</code>, I believe the right terminology should then become yeets</code>/Yeetable</code>/”re-yeet”. Personally I tend to care more about the system and framework we apply, than about the exact keyword we end up using.</p> Something that’s stood out while writing this is that the fallibility effect doesn’t seem to have a moral equivalent to IntoIterator</code>/IntoFuture</code> right now. I guess we have the automatic Into</code> conversion that is applied when we use ?</code>. But that is more general-purpose, and I’m having some trouble working through the exact implications of not having a separate trait.</p> Anyway, that’s it. I had a lot of fun writing this, and I hope it makes for an interesting read. Happy December!</p> Placing Arguments Wed, 13 Aug 2025 00:00:00 +0000 Introduction</h2> In my last post I introduced placing functions</a>, which are functions that can “return” values without copying them. This is useful not just because it’s more efficient (fewer copies), but also because it guarantees addresses remain stable - which is something we need for types that have internal borrows (self-referential types). Here is a little reminder of what placing functions look like:</p> #[</span>placing</span>] </span>// ← 1. Marks a function as "placing". </span>fn </span>new_cat</span>(</span>age</span>: </span>u8</span>) -> Cat { </span>// ← 2. Has a logical return type of `Cat`. </span> Cat { age } </span>// ← 3. Constructs `Cat` in the caller's frame. </span>} </span></code></pre> As you can see this is just regular Rust code, with the only difference being the added #[placing]</code> (placeholder) notation. The main idea behind placing functions is backwards-compatibility</em>, meaning: we should be able to progressively add #[placing]</code> notations to the entire stdlib, just like we’re doing with const. Which is important, because we don’t want to add yet another axis to the language, locking out address-sensitive types from being used with existing traits and functions. We’re already suffering through the bifurcation Pin</code> has introduced</a> 1</a></sup>, and I don’t think we should go through this again for self-referential types.</p> ^{1</sup> And no, unfortunately the “pin ergonomics” experiment won’t do anything to resolve this problem. I think it’s a shame we’re not taking this problem more seriously, because this kind of incompatibility affects the language in ways that extend far beyond syntax and conveniences.</p> </div>}Placing Functions</h2> It’s cool to have ideals and visions for how thing should be, but we do always need to square that with reality. And while placing functions</em> seem like they will work without a hitch (fallible placing functions post tbd), it’s placing arguments</em> which seem like they’re going to be a problem. To recap what placing arguments are about, the idea is that we would be able to write the following definition, and things would just work</em> without any copies:</p> impl</span><T> Box<T> { </span> </span>// `Box::new` here takes type `T` and </span> </span>// constructs it in-place on the heap </span> </span>fn </span>new</span>(</span>x</span>: #[placing] T) -> </span>Self </span>{ ... } </span>} </span></code></pre> In the post I made the following assertion (in bold even, because it’s important):</p> As a core design constraint: invoking a function that takes placing arguments should be no different from a regular function. This is needed to keep APIs backwards-compatible.</p> </blockquote> Cool idea, just one small hitch: we can’t actually do that. And that is because placing arguments</em> fundamentally encode callback semantics</em>. Olivier Faure provided this example to prove why:</p> let</span> x = Box::new({ </span> </span>return </span>0</span>; </span> </span>12 </span>}); </span></code></pre> To preserve compatibility, we need to preserve order of execution. In Rust today if you call this, the expression in Box::new</code> will be evaluated before Box::new</code> is called. In this case that means that this will return before Box::new</code> has a chance to allocate.</p> If we applied placing semantics, it would fundamentally need to change the ordering. Before we can place, the place has to be created. In this case that place is on the heap, which means interacting with the allocator, which in Rust can panic. That means that code which looks like it should return</code> before it calls the function, would actually panic. And that’s bad!</p> To fully push this point home, Taylor Cramer recently provided an example which showed how the execution ordering also causes issues with the borrow checker. In this example we would need to hold &vec</code> as part of the expression for vec.len</code>, while also holding &mut vec</code> for vec.push</code>. And that can’t work; not even with view types and abstract fields</a> 2</a></sup>:</p> ^{2</sup> Both Vec::len</code> and Vec::push</code> need to access the vector’s “length”. Virtualizing fields / partial borrows won’t change that.</p> </div>} vec.</span>push</span>(</span>make_big_thing</span>(vec.</span>len</span>())); </span>// ^^^^^^^^ ^^^^^^^ </span>// \| &mut vec \| &vec </span></code></pre> Closures</h2> The only way I see us making sense of this is by actually requiring that users use closures here. That would make the ordering far clearer, and preventing any surprises about ordering or borrows. Ralf Jung made the observation that FnOnce</code> for these kinds of scenarios actually perfectly encodes the semantics we want, and I agree. Consider this example:</p> vec.</span>push</span>(\|\| </span>make_big_thing</span>(vec.</span>len</span>())); </span></code></pre> This would give the following error:</p> error[E0502]: cannot borrow `vec` as mutable because it is also borrowed as immutable </span> --> src/main.rs:50:9 </span> \| </span>47 \| vec.push(\|\| make_big_thing(vec.len())); </span> \| --- immutable borrow occurs here </span>... </span>47 \| vec.push(\|\| make_big_thing(vec.len())); </span> \| ^^^ mutable borrow occurs here </span></code></pre> There is nothing specific to placing functions about this error, just like there’s nothing special about solving it:</p> let</span> len = vec.</span>len</span>(); </span>vec.</span>push</span>(\|\| </span>make_big_thing</span>(len)); </span></code></pre> But we can’t just change the signature of Vec::push</code> to take closures. Instead we would need to introduce a new method, e.g. Vec::push_with</code> that can take a closure and place its outputs:</p> impl</span><T> Vec<T> { </span> </span>pub fn </span>push_with</span><F>(&</span>mut </span>self</span>, </span>f</span>: F) </span> </span>where </span> F: #[placing] FnOnce() -> T; </span>} </span></code></pre> And this points us to a rather important challenge with these APIs, because for all intents and purposes this sets us down the path of deprecating Vec::push</code>. The Vec::push_with</code> method is more efficient than Vec::push</code>, and beyond some compat reasons there will be no reason to keep using Vec::push</code>. So people will naturally begin treating Vec::push</code> as legacy, even if we don’t outright mark it as such.</p> Edition-dependent path resolution</h2> I’m a firm believer that the obvious choice for something should be the right choice ^{3</a></sup>. It feels bad to scold people for using Box::new</code>, telling the that they should be using Box::new_with</code> instead. Or instead of calling Hashmap::insert</code>, they should be calling Hashmap::insert_with</code>. And so on.</p>} ^3</sup>While not quite the same, you can see this same idea of “the default way should be the right way” reflected in the safety practice of poka-yoke</a>. Or in Hollnagel’s distinction between Safety 1 and Safety 2</a>.</p> </div> My preferred way of resolving this would be to have a notion of edition-dependent paths</em>: imports and symbols that resolve differently depending on which edition you’re compiling on. For example on edition 2024</code> and below, people would see both Vec::push</code> and Vec::push_with</code>:</p> impl</span><T> Vec<T> { </span> </span>pub fn </span>push</span>(&</span>mut </span>self</span>, </span>item</span>: T); </span> </span>pub fn </span>push_with</span>(&</span>mut </span>self</span>, </span>f</span>: impl #[placing] FnOnce() -> T); </span>} </span></code></pre> But on edition 2024 + 1</code> we would be able to deprecate Vec::push</code> and rename it to something else (e.g. push_without</code>), and have Vec::push_with</code> take its place:</p> impl</span><T> Vec<T> { </span> </span>// Is called `push_with` on editions 2024 and below </span> </span>pub fn </span>push</span>(&</span>mut </span>self</span>, </span>f</span>: impl #[placing] FnOnce() -> T); </span> </span> </span>// Is called `push` on editions 2024 and below </span> #[</span>deprecated</span>(note = "</span>please use `push` instead</span>")] </span> </span>pub fn </span>push_without</span>(&</span>mut </span>self</span>, </span>item</span>: T); </span>} </span></code></pre> Under the hood these functions would still resolve to the same symbols after accounting for editions. Conceptually what this needs is a stable, edition-independent identifier that libraries can map back onto in edition-specific ways:</p> stable identifier</th> edition 2024</th> edition 2024 + 1</th></tr></thead> ::std::vec::Vec::push</code></td> Vec::push</code></td> Vec::push_without</code> †</td></tr> ::std::vec::Vec::push_with</code></td> Vec::push_with</code> †</td> Vec::push</code></td></tr> </tbody></table> †: These names are for illustrative purposes only; they are not concrete proposals.</em></p> And we could even go further with this, where in a hypothetical edition 2024 + 2</code> we might remove the old API entirely, so you would only ever be able to use the new API:</p> impl</span><T> Vec<T> { </span> </span>// Is called `push_with` on editions 2024 and below; </span> </span>// `push_without` is no longer available. </span> </span>pub fn </span>push</span>(&</span>mut </span>self</span>, </span>f</span>: impl #[placing] FnOnce() -> T); </span>} </span></code></pre> The main reason why we don’t have a system like this already is because it would be a lot of work and there are guaranteed to be edge cases that make this harder than you’d assume it is. The basic design I’m describing here is something people have thought of before, and that’s not the reason it hasn’t happened yet.</p> However useful edition-dependent path resolution would be in the long run, we don’t need it out of the gate. We can begin by adding placing variants of existing methods to the stdlib. It’s probably enough to proceed with, knowing edition-dependent path resolution might be possible in the future.</p> Thanks to Alice Ryhl for reviewing this post.</em></p> placing functions Tue, 08 Jul 2025 00:00:00 +0000 What are placing functions?</h2> About a year ago I observed that in-place construction seems surprisingly simple</a>. By separating the creating of the place in memory from writing the value to memory, it’s not that hard to see how we can turn that into a language feature. So about six months ago, that’s what I went ahead and did and created the placing crate</a>: a proc-macro-based prototype for “placing functions”.</p> Placing functions are functions whose return type is constructed in the caller’s stack frame rather than in the function’s stack frame. This means that the address from the moment on construction is stable</em>. Which may not only lead to improved performance, it also serves as the foundation for a number of useful features like supporting dyn</code> AFITs (Async Functions in Traits) 1</a></sup>.</p> In this post I’ll be explaining how placing functions desugar, why placing functions are the right solution for emplacement, and how placing functions integrate into the language. Not in as much detail as I’d do for an actual RFC, but more as a broad introduction to the idea</em> of placing functions. And to get right into it, here is a basic example of how this would work:</p> struct </span>Cat { </span> </span>age</span>: </span>u8</span>, </span>} </span> </span>impl </span>Cat { </span> #[</span>placing</span>] </span>// ← Marks a function as "placing". </span> </span>fn </span>new</span>(</span>age</span>: </span>u8</span>) -> </span>Self </span>{ </span> </span>Self </span>{ age } </span>// ← Constructs `Self` in the caller's frame. </span> } </span> </span> </span>fn </span>age</span>(&</span>self</span>) -> &</span>u8 </span>{ </span> &</span>self</span>.age </span> } </span>} </span> </span>fn </span>main</span>() { </span> </span>let</span> cat = Cat::new(</span>12</span>); </span>// ← `Cat` is constructed in-place. </span> assert_eq!(cat.</span>age</span>(), &</span>12</span>); </span>} </span></code></pre> A basic desugaring</h2> The purpose of the placing</code> crate is to prove that placing functions should not be that hard to implement. I managed to implement a working prototype in a few hours over my winter holidays. In total I've spent maybe four or so days on the implementation. I’m not a compiler engineer though, and I expect that the lovely folks on T-Compiler can probably recreate this in a fraction of the time it took me.</p> Since I don't know my way around the rustc frontend, I implemented the placing crate entirely using proc macros. The upside was that I could get something working more quickly. The downside is that proc macros don’t have access to type information, so I had to hack around that limitation. Which results in an API that requires a lot of proc macro attributes. But that’s ok for a proof of concept.</p> Let’s walk through the basic example I showed earlier, but this time using the proc macros. Starting by installing the placing crate:</p> $</span> cargo add placing </span></code></pre> We can then import placing</code> and define our main struct Cat</code>. We need to annotate this with the #[placing]</code> attribute macro because we need to change the internal representation slightly. Here is what this looks like:</p> //! Original </span> </span>use </span>placing::placing; </span> </span>#[</span>placing</span>] </span>pub struct </span>Cat { </span> </span>age</span>: </span>u8</span>, </span>} </span></code></pre> Let’s take a look at what the #[placing]</code> annotation expands to. As I said in the intro: placing functions separate the creation of the memory location from the initialization of the values in said location. For our type Cat</code>, the location must be of type MaybeUninit<Cat></code>. But because we want to keep the type the same, even if we change how the internals work, we actually want to keep Cat</code> as the outer type name and move the fields into an internal MaybeUninit</code>:</p> //! Desugared </span> </span>use </span>std::mem::MaybeUninit; </span> </span>/// This keeps the same external type, </span>/// but changes the internals to store the fields </span>/// in a `MaybeUninit`. </span>#[</span>repr</span>(transparent)] </span>pub struct </span>Cat(MaybeUninit<InnerCat>); </span> </span>/// These are the fields contained in the original </span>/// type `Cat`. But separated so that they can be </span>/// wrapped in a `MaybeUninit` internally. </span>struct </span>InnerCat { </span> </span>age</span>: </span>u8</span>, </span>} </span></code></pre> Our desugaring needs to add one last bit to our type definition to ensure it works correctly. Because we are now holding a MaybeUninit</code> we have to make sure it calls its destructors when dropped. This means our desugaring needs to generate a Drop</code> implementation that calls through the MaybeUninit</code>.</p> //! Desugared </span> </span>impl </span>Drop </span>for </span>Cat { </span> </span>fn </span>drop</span>(&</span>mut </span>self</span>) { </span> </span>// The constructors guarantee this will never </span> </span>// be dropped before it has been initialized. </span> </span>unsafe </span>{ </span>self</span>.</span>0.</span>assume_init_drop</span>() } </span> } </span>} </span></code></pre> Now that we have our type definition, let’s show how to implement the new</code> constructor. Because we don’t have access to type information the placing crate requires annotations on both the impl block and the method:</p> //! Original </span> </span>#[</span>placing</span>] </span>impl </span>Cat { </span> #[</span>placing</span>] </span> </span>fn </span>new</span>(</span>age</span>: </span>u8</span>) -> </span>Self </span>{ </span> </span>Self </span>{ age } </span> } </span>} </span></code></pre> This is the trickiest part of the desugaring because it needs to split up the constructor into two parts. One to create the place, the other to initialize the values in-place. Conceptually that means rewriting the return type of the last line into writing into a mutable argument instead.</p> //! Desugared </span> </span>use </span>std::mem::MaybeUninit; </span> </span>impl </span>Cat { </span> </span>/// Calling this function constructs the place to </span> </span>/// initialize the type into. This is part of a </span> </span>/// two-part constructor, and it must always be </span> </span>/// followed by a call to `new_init`. </span> </span>unsafe fn </span>new_uninit</span>() -> </span>Self </span>{ </span> </span>Self</span>(MaybeUninit::uninit()) </span> } </span> </span> </span>/// This initializes the values of the type in-place. </span> </span>/// `new_init` must not be called more than once, and </span> </span>/// must always follow a call to `new_uninit`. </span> </span>unsafe fn </span>new_init</span>(&</span>mut </span>self</span>, </span>age</span>: </span>u8</span>) { </span> </span>let</span> this = </span>self</span>.</span>0.</span>as_mut_ptr</span>(); </span> </span>unsafe </span>{ (&raw </span>mut </span>(this).age).</span>write</span>(age) }; </span> } </span>} </span></code></pre> Next we also have a getter function. All we need to do with that is teach it how to reach through the outer struct and into the inner fields. Here is the definition:</p> //! Original </span> </span>#[</span>placing</span>] </span>impl </span>Cat { </span> </span>fn </span>age</span>(&</span>self</span>) -> </span>u8 </span>{ </span> &</span>self</span>.age </span> } </span>} </span></code></pre> And here is what it expands to:</p> //! Desugared </span> </span>impl </span>Cat { </span> </span>fn </span>age</span>(&</span>self</span>) -> </span>u8 </span>{ </span> </span>let</span> this = </span>unsafe </span>{ </span>self</span>.</span>0.</span>assume_init_ref</span>() }; </span> this.age </span> } </span>} </span></code></pre> With that we’re now ready to create an instance of Cat</code> in-place, and invoke our getter. This is the only part that can’t be abstracted away, since Rust macros have very strict scoping rules compared to if we implemented this directly in the compiler. What we really wish we could do would be this:</p> let</span> cat = placing!(Cat, new, </span>12</span>); </span></code></pre> But for now we have to call this manually instead, and so the invocation looks like this:</p> //! Original </span> </span>fn </span>main</span>() { </span> </span>let mut</span> cat = </span>unsafe </span>{ Cat::new_uninit() }; </span> </span>unsafe </span>{ cat.</span>new_init</span>(</span>12</span>) }; </span> assert_eq!(cat.</span>age</span>(), &</span>12</span>); </span>} </span></code></pre> As an aside: in Rust we now have the experimental super_let</code> feature which makes it possible to construct types in the enclosing scope, but reference them afterwards. This almost</em> works for our use case, except it can only return references, not owned types. That means the best we can do with that feature is the following (playground</a>):</p> #![</span>feature</span>(super_let)] </span> </span>macro_rules! </span>new_cat { </span> (</span>$value</span>:</span>expr </span>$(,)?) => { </span> { </span> </span>super let mut</span> cat = </span>unsafe </span>{ Cat::new_uninit()) }; </span> </span>unsafe </span>{ Cat::new_init(&</span>mut</span> cat, </span>$value</span>) }; </span> &</span>mut</span> cat </span>// ← ❌ Returns by-ref rather than by-value </span> } </span> } </span>} </span></code></pre> If we try and return an owned value it actually copies it - which is exactly what we’re trying to avoid. We might be able to change that in the compiler implementation. And to summarize: here is the placing</code> crate’s version of our original example:</p> //! Original </span> </span>use </span>placing::placing; </span> </span>#[</span>placing</span>] </span>struct </span>Cat { </span> </span>age</span>: </span>u8</span>, </span>} </span> </span>#[</span>placing</span>] </span>impl </span>Cat { </span> #[</span>placing</span>] </span> </span>fn </span>new</span>(</span>age</span>: </span>u8</span>) -> </span>Self </span>{ </span> </span>Self </span>{ age } </span> } </span> </span> </span>fn </span>age</span>(&</span>self</span>) -> </span>u8 </span>{ </span> &</span>self</span>.age </span> } </span>} </span> </span>fn </span>main</span>() { </span> </span>// `Cat` is constructed in-place. </span> </span>let mut</span> cat = </span>unsafe </span>{ Cat::new_uninit() }; </span> </span>unsafe </span>{ cat.</span>new_init</span>() }; </span> </span> assert_eq!(cat.</span>age</span>(), &</span>12</span>); </span>} </span></code></pre> And here is the same code with all the macros expanded:</p> //! Desugared </span> </span>use </span>std::mem::MaybeUninit; </span> </span>#[</span>repr</span>(transparent)] </span>struct </span>Cat(MaybeUninit<InnerCat>); </span>struct </span>InnerCat { </span> </span>age</span>: </span>u8</span>, </span>} </span> </span>impl </span>Cat { </span> </span>/// Creates an uninitialized place </span> </span>unsafe fn </span>new_uninit</span>() -> </span>Self </span>{ </span> </span>Self</span>(MaybeUninit::uninit()) </span> } </span> </span> </span>/// Initializes the fields in-place </span> </span>fn </span>new_init</span>(&</span>mut </span>self</span>, </span>age</span>: </span>u8</span>) { </span> </span>let</span> this = </span>self</span>.</span>0.</span>as_mut_ptr</span>(); </span> </span>unsafe </span>{ (&raw </span>mut </span>(this).age).</span>write</span>(age) }; </span> } </span> </span> </span>fn </span>age</span>(&</span>self</span>) -> </span>u8 </span>{ </span> </span>let</span> this = </span>unsafe </span>{ </span>self</span>.</span>0.</span>assume_init_ref</span>() }; </span> this.age </span> } </span>} </span> </span>impl </span>Drop </span>for </span>Cat { </span> </span>fn </span>drop</span>(&</span>mut </span>self</span>) { </span> </span>unsafe </span>{ </span>self</span>.</span>0.</span>assume_init_drop</span>() } </span> } </span>} </span> </span>fn </span>main</span>() { </span> </span>let mut</span> cat = </span>unsafe </span>{ Cat::new_uninit() }; </span> </span>unsafe </span>{ cat.</span>new_init</span>() }; </span> assert_eq!(cat.</span>age</span>(), &</span>12</span>); </span>} </span></code></pre> The placing crate also supports desugaring types that return Result</code>, Box</code>, and Arc</code>. As well as includes support for nesting constructors, where you end up calling a #[placing] fn</code> from a #[placing] fn</code>. This is why I’m fairly confident this should end up working out.</p> The main limitation of the crate is that it doesn’t yet support traits. I started adding support for that, but ended up running out of time. This is the reason why if you look at the codegen in the crate you’ll see a PLACING: bool</code> const generic inserted everywhere. It isn’t particularly useful yet, but now you know why it’s there.</p> Thinking in placing functions</h2> Placing functions draw their inspiration from two other language features:</p> Super Let (Rust Nightly)</strong>: is an experimental feature that enables temporary lifetime extensions</a>. This allows variables to be created in the enclosing scope.</li> Guaranteed Copy Elision (C++ 17)</strong>: sometimes also called “deferred temporary materialization”</a> guarantees that structs are always constructed in the caller’s scope.</li> </ul> If super let</code> operates on block scopes, you can think of placing functions as operating across function boundaries. And if guaranteed copy elision is an automated guarantee that applies to all</em> functions, you can think of placing functions as only ever applying to functions that have been explicitly opted into this feature.</p> The design of placing functions attempts to balance three core constraints:</p> Control</strong>: in certain cases emplacement already happens through optimizations (e.g. inlining). An emplacement language feature must guarantee</em> it emplaces, or else fail compilation.</li> Integration</strong>: C++’s guaranteed copy elision shows just how broadly applicable emplacement really is. That means the initial upper bound for this language feature is every single function with a return type. That’s incredibly broad, and we need to ensure it integrates closely and easily with the rest of the language.</li> Compatibility</strong>: Rust’s stdlib makes strong backwards-compatibility guarantees. We will want it to be able to make use of emplacement without</em> breaking backwards compat. This means we cannot mint new traits just to support emplacement, or add new APIs specifically to emplace.</li> </ul> It would be a mistake to think of emplacement as a feature with narrow applicability; C++ provides evidence emplacement is relevant to almost every constructor</strong>. The right way to think about emplacement is to assume it has a maximally broad upper bound. But then start designing and implementing a minimal subset. While we may eventually find limitations or cases where emplacement isn’t possible; that will be something we prove out through implementation.</p> This is why I believe we should model “emplacement” more like an effect, and not like a different kind of language feature. I think of placing functions as somewhere between const functions and async functions. They change the codegen of the function, not entirely unlike the generator transform we use for async</code> and gen</code>. But it never actually lowers to another type we can observe in the type system, which makes it very similar to const</code>.</p> Prior Art in Rust</h2> As I was getting ready to publish this post I ended up talking a little more with Sy Brand about placing functions, C++, and ABIs. It turns out: Rust already guarantees emplacement in a number of cases. Consider for example the following code:</p> pub struct </span>A { </span> </span>a</span>: </span>i64</span>, </span> </span>b</span>: </span>i64</span>, </span> </span>c</span>: </span>i64</span>, </span> </span>d</span>: </span>i64</span>, </span> </span>e</span>: </span>i64</span>, </span>} </span> </span>impl </span>A { </span> </span>pub fn </span>new</span>() -> A { </span> A { </span> a: </span>42</span>, </span> b: </span>69</span>, </span> c: </span>4269</span>, </span> d: </span>6942</span>, </span> e: </span>696942</span>, </span> } </span> } </span>} </span></code></pre> When we compile this for the SYSV ABI on x86, it outputs the following assembly (godbolt</a>):</p> example::A::new::hd00831bc57a4b613: </span> </span>mov </span>rax</span>, </span>rdi </span> </span>mov </span>qword ptr </span>[</span>rdi</span>], </span>42 </span> </span>mov </span>qword ptr </span>[</span>rdi </span>+ </span>8</span>], </span>69 </span> </span>mov </span>qword ptr </span>[</span>rdi </span>+ </span>16</span>], </span>4269 </span> </span>mov </span>qword ptr </span>[</span>rdi </span>+ </span>24</span>], </span>6942 </span> </span>mov </span>qword ptr </span>[</span>rdi </span>+ </span>32</span>], </span>696942 </span> </span>ret </span></code></pre> This assembly is writing directly to pointer offsets provided to the function. In other words: this function is emplacing. And it’s actually guaranteed to do that, as defined in the x86 SYSV ABI spec</a>, section 3.2.3:</p> [!quote] If the type has class MEMORY, then the caller provides space for the return value and passes the address of this storage in %rdi</code> as if it were the first argument to the function. In effect, this address becomes a “hidden” first argument This storage must not overlap any data visible to the callee through other names than this argument. On return %rax</code> will contain the address that has been passed in by the caller in %rdi</code>.</p> </blockquote> A hidden first argument that’s passed to functions? That sure sounds a lot like how the desugaring for placing functions is intended to work. In fact, C++’s guaranteed copy elision makes use of this very same feature. Section 3.2.3 states the following:</p> If a C++ object has either a non-trivial copy constructor or a non-trivial destructor 11, it is passed by invisible reference (the object is replaced in the parameter list by a pointer that has class INTEGER) 12.</p> </blockquote> This is all incredibly similar to what I’m proposing, but happening automatically at the ABI level rather than transparently at the language level. It also begs the question: how easy it would be to modify rustc's ABI lowering code for x64 to just say "if the type was declared with the placing</code> attribute, it's always classified as MEMORY”</em>?</p> Q&A</h2> What about placing arguments?</h3> So far this post has only discussed placing in relation to return types. For our goal of preserving express compatibility with the existing stdlib, placing return types are not enough. Back in March Eric Holk and Tyler Mandry argued that we’ll also want to have some form of placing arguments</em> (or at least the capability that enables us to do this). So let’s use Box::new</code> as our example to show why. Without placing arguments, the best we could do would be to define some form of Box::new_with</code> function that takes a placing closure:</p> impl</span><T> Box<T> { </span> </span>// The existing default constructor </span> </span>fn </span>new</span>(</span>x</span>: T) -> </span>Self </span>{ ... } </span> </span> </span>// The newly introduced `placing` constructor </span> </span>fn </span>new_with</span><F>(</span>f</span>: F) -> </span>Self </span> </span>where </span> F: #[placing] FnOnce() -> T, </span> { ... } </span>} </span></code></pre> The new_with</code> constructor is always preferable to the new</code> constructor because it guarantees the absence of intermediate stack copies. This will lead to an effective deprecation Box::new</code>. If not outright, then likely first by way of ”best practices”</em>.</p> The way to solve this would be to enable Box::new</code> to act as the receiver of values which need to be emplaced. This would be done by requiring annotations not at the function level, but at the argument/return-type level. Keeping with our #[placing]</code> placeholder notation, we can imagine it looking something like this:</p> //! Original </span> </span>impl</span><T> Box<T> { </span> </span>// `Box::new` here takes type `T` and </span> </span>// constructs it in-place on the heap </span> </span>fn </span>new</span>(</span>x</span>: #[placing] T) -> </span>Self </span>{ ... } </span>} </span></code></pre> I expect the desugaring of this will likely look somewhat similar to Alice Ryhl’s in-place initialization RFC</a>, desugaring to some form of impl Emplace</code> trait. But crucially: this would only be observable within the implementation, and not to any of the callers.</p> //! Desugared </span> </span>/// Write a value to a place. </span>trait </span>Emplace<T> { </span> </span>fn </span>emplace</span>(</span>self</span>, </span>slot</span>: </span>mut</span> T); </span>} </span> </span>impl</span><T> Box<T> { </span> </span>fn </span>new</span>(</span>x</span>: impl Emplace<T>) -> </span>Self </span>{ </span> </span>let mut</span> this = Box::<T>::new_uninit(); </span>// 1. create the place </span> x.</span>emplace</span>(this.</span>as_mut_ptr</span>()); </span>// 2. init the value </span> Ok(this.</span>assume_init</span>()) </span>// 3. all done </span> } </span>} </span></code></pre> Now the reason why I’ve put this under Q&A is because I haven’t yet figured out the finer language rules here since I haven’t implemented this yet. As a core design constraint: invoking a function that takes placing arguments should be no different from a regular function. This is needed to keep APIs backwards-compatible.</strong> What should make this special is that functions with placing arguments and placing return types should work together to emplace.</p> What about borrows / local lifetime extensions?</h3> In her post on super let</a>, Mara provides a clear example for when temporary lifetime extensions are useful. Here Writer::new</code> takes an &'a File</code>, and we need a feature like super let</code> to create an instance of File</code> that outlives the scope of the block:</p> let</span> writer = { </span> println!("</span>opening file...</span>"); </span> </span>let</span> filename = "</span>hello.txt</span>"; </span> </span>super let</span> file = File::create(filename).</span>unwrap</span>(); </span> Writer::new(&file) </span>}; </span></code></pre> The return type Writer</code> here must have a lifetime to be able to reference super let file</code>. But it can’t be a normal lifetime, since it doesn’t adhere to the usual rules. Without specifying any of the concrete rules, this lifetime has been dubbed 'super</code>. From the perspective of the block this behaves not unlike 'static</code> - though crucially it is not the same as 'static</code>.</p> Now the question is: how can we represent this block as a function instead? Because it makes sense that we would want to eventually be able to factor out functionality from blocks into functions. We’d probably want to do that using a 'super</code> lifetime, like so:</p> //! Original </span> </span>fn </span>create_writer</span>(</span>filename</span>: &</span>str</span>) -> Writer<</span>'super</span>> { </span> println!("</span>opening file...</span>"); </span> </span>super let</span> file = File::create(filename).</span>unwrap</span>(); </span> Writer::new(&file) </span>} </span></code></pre> Note how Writer</code> itself does not require a #[placing]</code> annotation: it’s okay that we copy it out of the function. The only important part is that file</code> outlives the current scope. The desugaring for this is quite fun, even if we can’t yet represent it in the type system. What we need to do here is ensure that file</code> is constructed in-place in the caller’s scope. And once initialized we can reference it using a blank/unsafe lifetime in our return type. I haven’t actually checked this, but I believe this is a valid desugaring:</p> //! Desugared </span> </span>fn </span>create_writer</span><</span>'a</span>>(</span>filename</span>: &</span>str</span>, </span>file</span>: &</span>'a mut </span>MaybeUninit<File>) -> Writer<</span>'a</span>> { </span> println!("</span>opening file...</span>"); </span> </span>let</span> file = </span>unsafe </span>{ file.</span>write</span>(File::create(filename).</span>unwrap</span>()) }; </span> Writer::new(file) </span>} </span></code></pre> This however needs to be paired with a function prelude on invocation to create the place for file</code>. We can therefore imagine the invocation of this function looking something like this:</p> // with syntactic sugar </span>let</span> writer = </span>create_writer</span>("</span>hello.text</span>"); </span> </span>// desugared </span>let mut</span> file = MaybeUninit::uninit(); </span>let</span> writer = </span>create_writer</span>("</span>hello.text</span>", &</span>mut</span> file); </span></code></pre> What about pinning?</h3> Once we have emplacement in the language, most of the reasons for Pin</code> being the way it is kind of fall away. But there is a gap between having emplacement, and then also having !Move</code>, and so we do need to have some form of compatibility with Pin</code>. Luckily the Pin</code> type is just a special-case of the previous lifetime extension example.</p> What’s neat about placing functions is that it would allow us to replace the std::pin::pin!</code> macro with a pin</code> free-function, using the 'super</code> lifetime:</p> //! Original </span> </span>pub fn </span>pin</span><T>(</span>t</span>: T) -> Pin<&</span>'super mut</span> T> { </span> </span>super let mut</span> t = t; </span> </span>unsafe </span>{ Pin::new_uninit(&</span>mut</span> t) } </span>} </span></code></pre> All the desugaring needs to do to make this work is change the function to take an additional slot MaybeUninit<T></code> that we can write our value into. This allows us to extend the lifetime, after which we can reference it in our return type like so:</p> //! Desugared </span> </span>pub fn </span>pin</span><T, </span>'a</span>>(</span>t</span>: T, </span>slot</span>: &</span>'a mut </span>MaybeUninit<T>) -> Pin<&</span>'a mut</span> T> { </span> </span>let mut</span> t = </span>unsafe </span>{ slot.</span>write</span>(t) }; </span> </span>unsafe </span>{ Pin::new_uninit(&</span>mut</span> t) } </span>} </span></code></pre> Are annotations necessary?</h3> RFC 2884: Placement by Return</a> proposed introducing C++’s Guaranteed Copy Elision rules to Rust almost as-is. I appreciate this RFC because it has the right idea about the scope of the changes, and does it solely by changing the meaning of return</code>. But where it runs into trouble is that it only</em> changes the meaning of return</code>. And so instead of adding placement to e.g. Box::new</code>, it needs to add a new method Box::new_with</code>.</p> Fundamentally there are three kinds of placement we’re interested in:</p> Placing return types</strong>: where we want a to avoid copying the type returned from a function. For example if we want a referentially-stable constructor.</li> Placing function arguments</strong>: where we want to avoid copying an argument passed into a function. For example: to when constructing a type on the heap.</li> Lifetime extensions</strong>: where we want to reference a local variable from a local type which will outlive the current scope (lifetime extension). For example: when pinning.</li> </ul> Even if C++’s Guaranteed Copy Elision should be our ultimate goal; annotations allow us to get there incrementally. Placing return types are fairly easy. Placing function arguments are a little harder. And lifetime extensions will be harder still. Being able to opt into this via explicit annotations means we can start small and gradually build up.</p> For something as broad as “functions with arguments or return types” that seems like the right way to start. And if for some reason this feature ends up being so successful we’ll want to annotate nearly every function with it, changing defaults seems like something that could be done over an edition if we wanted to.</p> What about self-referential types?</h3> I’ve written at length about self-referential types</a> before. Once we have placing functions, we end up with three components for generalized self-referential types are:</p> Placing functions</strong>: so we can construct a type in a stable position in memory</li> Self-lifetimes</strong>: so you can declare that some field borrows from some other field.</li> Partial constructors</strong>: so you can start by initializing the owned data first, and initialize the references to that data second.</li> </ol> We’ve already seen placing functions. Self-lifetimes would allow fields to refer to data contained in other fields, which would probably look something like this:</p> struct </span>Cat { </span> </span>data</span>: String, </span> </span>name</span>: &</span>'self</span>.data </span>str</span>, </span>// ← references `self.data` </span>} </span></code></pre> And partial constructors would allow you to construct types in multiple phases. For example, here we first initialize the field data</code> in Cat</code>. And then we take the string contained within, and do some crude parsing to interpret everything up until the first space as the cat’s name:</p> fn </span>new_cat</span>(</span>data</span>: String) -> Cat { </span> </span>let mut</span> cat = Cat { data, .. }; </span> cat.name = cat.data.</span>split</span>(' ').</span>next</span>().</span>unwrap</span>(); </span> cat </span>} </span></code></pre> Once we have this, we can of course combine it with the lifetime extension examples we’ve shown before to ensure that the type remains in a stable memory location. But crucially: this is also forward-compatible with alternative mechanisms for referential stability such as the Move</code> auto-trait.</p> As a side note: partial constructors are basically the same feature as view types and pattern types</a>. It’s still the same general refinement feature, but now with an added rule that we can go from a refinement type back to the original type by populating its fields. Assigment here takes the place of an inverse match</code> if you will.</p> What’s nice about this design is that these features are all orthogonal, but complimentary. Emplacement is useful even without partial initialization. And partial initialization (refinement) is useful even without self-referential lifetimes. Features complimenting each other in this way to me is the hallmark of good language design. It means it generalizes beyond just a niche use case. But simultaneously becomes even more useful when combined with other features.</p> Why not directly rely on Init<T></code> or &out</code>?</h3> Both the Init</code> type and &out</code> parameters feature are backwards-incompatible to add to existing types and interfaces. This is a problem, because placement is broadly applicable: we know from C++ 17 that virtually every constructor wants to be placing. And we can’t reasonably rewrite every function returning -> T</code> to instead return -> Init<T></code> or take &out T</code>.</p> // 1. Original signature </span>fn </span>new_cat</span>() -> Cat { ... } </span> </span>// 2. Using `Init`, changes the signature </span>fn </span>new_cat</span>() -> Init<Cat> { ... } </span> </span>// 3. Using `&out`, changes the signature </span>fn </span>new_cat</span>(</span>cat</span>: &out Cat) { ... } </span> </span>// 3. Using `#[placing]`, preserves the signature </span>#[</span>placing</span>] </span>fn </span>new_cat</span>() -> Cat { ... } </span></code></pre> That doesn’t mean that these designs are inherently broken or incorrect; far from it actually. But because they seem to assume a different scope for the design, it naturally means those designs are operating with a different set of design constraints - in turn leading to different designs.</p> I believe that RFC 2884: Placement by Return</a> by PoignardAzur had the right idea. In order for Rust to be competitive with C++, we need to be able to guarantee most constructors can emplace. And in order to do that, we can’t require people to rewrite their code.</p> However, the Init RFC</a> has some great ideas about how to emplace function arguments. Which is something that RFC 2884 didn’t have a good answer to. I believe that Rust’s strength lies in its ability to distill different ideas and synthesize them into something new. I believe that we can arrive at something truly great if we combine super let</code>, placement-by-return, Init</code>, and ensure it is backwards-compatible.</p> Can placing functions be nested?</h3> Yes - placing functions called in a return position should be able to compose. For the language feature should be relatively straight-forward when calling one placing function inside of another in the return position:</p> struct </span>Foo {} </span>#[</span>placing</span>] </span>fn </span>inner</span>() -> Foo { </span> Foo {} </span>// ← 1. Constructed in the caller's scope </span>} </span> </span>#[</span>placing</span>] </span>fn </span>outer</span>() -> Foo { </span> </span>inner</span>() </span>// ← 2. Forwards the emplacement to its caller </span>} </span></code></pre> This itself is reminiscent of C++’s guaranteed copy elision guarantees, which are composable through functions. That is because in C++ temporaries are only materialized into real objects at the end of a call chain, enabling arbitrarily deep composition. For Rust it’s important that we maintain this same property, including when wrapping and composing types:</p> struct </span>Foo {} </span>#[</span>placing</span>] </span>fn </span>inner</span>() -> Foo { </span> Foo {} </span>// ← 1. Constructed in the caller's scope </span>} </span> </span>struct </span>Bar(Foo) </span>#[</span>placing</span>] </span>fn </span>outer</span>() -> Bar { </span> Bar(</span>inner</span>()) </span>// ← 2. Emplaces Bar in the caller, and Foo in Bar </span>} </span></code></pre> In this example Bar</code> is constructed in the caller’s scope, and as part of initialization it invokes and emplaces Foo</code> inside of it. I stopped working on the placing crate before implementing this, but for this desugaring to work all we need to do is for the “place” used by the inner</code> function to by placed inside of the “place” used by the outer</code> function.</p> The most complex kind of composition is when we start involving temporaries. Consider the following example which constructs a type in the first function, mutates it in the second function, and finally uses it in a third function:</p> /// A Cat which is constructed in place and can meow. </span>struct </span>Cat {} </span>impl </span>Cat { </span> #[</span>placing</span>] </span> </span>fn </span>new</span>(</span>name</span>: String) -> </span>Self </span>{ .. } </span> </span>fn </span>set_name</span>(&</span>mut </span>self</span>) { .. } </span> </span>fn </span>meow</span>(&</span>self</span>) { .. } </span>} </span> </span>/// Construct a value. </span>#[</span>placing</span>] </span>fn </span>first</span>() -> Cat { </span> </span>let</span> name = "</span>Nori</span>".</span>to_string</span>(); </span> Cat::new(name) </span>// ← 1. Emplaced in the caller </span>} </span> </span>/// Mutate the value. </span>#[</span>placing</span>] </span>fn </span>second</span>() -> Cat { </span> </span>super let mut</span> cat = </span>first</span>(); </span>// ← 2. Emplaced in the caller </span> cat.</span>set_name</span>("</span>Chashu</span>".</span>to_string</span>()); </span>// ← 3. Mutated </span> cat </span>// ← 4. Logically returned </span>} </span> </span>/// Use the value. </span>fn </span>third</span>() { </span> </span>let</span> cat = </span>second</span>(); </span>// ← 5. Placed on-stack here </span> cat.</span>meow</span>(); </span>} </span></code></pre> We need to have some way to emplace temporaries that we later logically return from the caller. The super let</code> feature seems like it would particularly well for this. In the 'move</code> example we were returning a reference to a super let</code> value from a function. But I think it would make a lot of sense if we could use super let</code> to return logically owned values from a function too, as shown here.</p> Conclusion</h2> This post introduces a design for placing functions: a declarative addition to Rust that enables types to be constructed in-place. Unlike alternative APIs such as Init<T></code> and &out</code>, placing functions are designed to keep function signatures intact, enabling existing functions and APIs to be annotated as retroactively.</strong> In other words: placing functions were designed to prioritize backwards-compatibility.</p> The placing functionality has been prototyped in the placing crate</a>. Even though this crate was designed with limited time and entirely using procedural macros, it should be sufficient to prove the feasibility of a language feature.</p> While placing functions are the most important placement-related feature, they are not the only one. There are three kinds of placing functions total that need to be addressed:</p> Placing return types</strong>: where we want a to avoid copying the type returned from a function. For example if we want a referentially-stable constructor.</li> Lifetime extensions</strong>: where we want to reference a local variable from a local type which will outlive the current scope (lifetime extension). For example: when pinning.</li> Placing function arguments</strong>: where we want to avoid copying an argument passed into a function. For example: to when constructing a type on the heap.</li> </ol> A complete solution should address all three of these uses. Placing functions only directly enables placing return types, but in the discussion section we’ve also discussed how we could extend this to include lifetime extensions and placing function arguments.</p> When discussing emplacement it’s important to consider both intra-function and inter-function variants. The experimental super let</code> feature only works within</em> functions today (intra-function). With placing functions working very similarly, enabling similar functionality to work between</em> functions too (inter-function). A good design should make both variants easy, convenient, and interoperable.</p> In this post we’ve also explained why the scope for emplacement is broad: in C++ constructors guarantee emplacement by default. And given Rust wants to have performance that’s competitive with C++, we have to assume that most functions will eventually want to make that guarantee as well. And the only realistic way to achieve that is if we can guarantee emplacement in a backwards-compatible way.</p> Thanks to Sy Brand for reviewing earlier copies of this post and especially providing valuable feedback and clarifications about both C++’s copy elisions feature, as well as how the SYSV ABI works.</em></p> ^{1</sup> I only had a hunch this was possible. Alice Ryhl has done the work to prove this can be done. Albeit for a slightly different, but very similar proposal.</p> </div>} Tree-Structured Concurrency II: Replacing Background Tasks With Actors Wed, 02 Jul 2025 00:00:00 +0000 Structured concurrency</h2> (Tree-)Structured Concurrency</a> is neat because it greatly simplifies concurrent programs. It greatly reduces, if not outright eliminates the possibility of logical races due to concurrency issues. And conceptually it’s not that hard either, as we can encode structured concurrency with just two rules:</p> Every child computation (except the root) must</em> have exactly one logical parent at any given time during execution.</li> Every parent computation may</em> have multiple child computations executing concurrently at any given moment.</li> </ol> Every child must always have exactly one parent. But every parent may have multiple children. These two rules allow us to infer a third rule that is at the core of the system:</p> Child computations will never outlive their parents.</li> </ol> If we apply these rules, we’ll notice that the call-graph of a program will naturally form a tree. This is nothing special, since non-concurrent programs already work this way. If that’s not intuitive: realize that for example flame graphs and flame charts are nothing more than a visualization of tree-shaped data.</p> The background task problem</h2> One of the most common questions people ask when learning about structured concurrency is:</p> “How do I run work after</em> returning from a function?”</p> </blockquote> This is a fair question! A common example is when implementing an HTTP server: you may want to log metrics on every request, but you don’t want to delay sending the response until logging has completed. At which point people often reach for unstructured concurrency in the form of background tasks</em>, often using some form of task::spawn</code>:</p> /// Some HTTP handler </span>async </span>fn </span>handler</span>(</span>req</span>: Request, </span>_context</span>: ()) -> Result<Response> { </span> </span>// Get the body from a request </span> </span>let</span> body = req.</span>body</span>().</span>try_to_string</span>()?; </span> </span> </span>// Create a detached background task that </span> </span>// does some work in the background </span> task::spawn(async { </span> </span>background_work</span>(body).await.</span>unwrap</span>(); </span> }); </span> </span> </span>// Construct a response and return </span> </span>// it from the handler </span> Ok(Response::Ok()) </span>} </span></code></pre> In this example, task::spawn</code> creates a background task which is not owned by any parent. This violates the second rule of structured concurrency: every computation should always have one logical parent. We can see this being a problem if background_work</code> were to complete unsuccessfully. Right now if it fails we have no way to propagate the error or retry the operation, so all we can realistically do is let it crash.</p> In this example we also have no way of cancelling the work done in the task since it’s not rooted in any logical parent. But despite those issues, use case itself is also a very reasonable one: we shouldn’t have to wait for background work to finish before sending off a response. So: what do we do?</p> The actor pattern</h2> What we want is for work to happen in the background, while simultaneously also being rooted in some parent computation. I believe the right way to solve this is by using actors: types that we can send data and will schedule work for us.</p> In frameworks like actix</a> and choo</a> this is done using messages</em> that are dispatched using channels or event emitters. But languages like Swift have actor support</a> built directly into the language, and rely on methods instead of messages.</p> But for what we’re trying to do here we don’t really need any fancy libraries or frameworks: we can easily build our own actors using structs, impl blocks, and channels. Here is a basic template you can use to implement your own actors with. Without comments this comes out to maybe 15 lines total:</p> // Using the `async-channel` and </span>// `futures-concurrency` libraries </span>use </span>async_channel::{</span>self </span>as channel, Sender, Receiver}; </span>use </span>futures_concurrency::prelude::; </span> </span>/// The type of data we will send to our actor. </span>/// This is just a simple alias to our HTTP </span>/// framework's `Body` type. </span>type </span>Message = Body; </span> </span>/// Our custom actor type </span>struct </span>Actor(Receiver<Message>); </span> </span>/// A handle to the actor which can be </span>/// used to schedule messages with. </span>type </span>Handle = Sender<Message>; </span> </span>impl </span>Actor { </span> </span>/// Create a new `Actor` instance </span> </span>fn </span>new</span>(</span>capacity</span>: </span>usize</span>) -> (</span>Self</span>, Handle) { </span> </span>let </span>(sender, receiver) = channel::bounded(capacity); </span> (</span>Self</span>(receiver), sender) </span> } </span> </span> </span>/// Start listening for incoming messages </span> </span>/// and handle them concurrently </span> async </span>fn </span>run</span>(&</span>mut </span>self</span>) -> Result<()> { </span> </span>// Using `ConcurrentStream` from `futures-concurrency` </span> </span>// to take work from the channel and execute it </span> </span>// concurrently. To bound the amount of concurrency, </span> </span>// you can use the provided `.limit` combinator. </span> </span>self</span>.</span>0.</span>co</span>().</span>try_for_each</span>(\|</span>msg</span>\| { </span> </span>// Work is scheduled here </span> </span>background_work</span>(body).await?; </span> Ok(()) </span> }).await?; </span> Ok(()) </span> } </span>} </span></code></pre> Now to use this we need to create an instance, run it concurrently with our HTTP server, and make sure request handlers in the server have access to it. I’m most familiar with the Tide HTTP framework</a> so I’ll be using that here, but your favorite framework probably allows you to do something similar. Here is the way to tie this all together:</p> use </span>futures_concurrency::prelude::; </span> </span>#[</span>async_main</span>] </span>async </span>fn </span>main</span>() -> server::Result<()> { </span> </span>// Create an instance of our actor </span> </span>let </span>(actor, handle) = Actor::new(); </span> </span> </span>// Create an instance of our server </span> </span>// and pass it the actor handle </span> </span>let mut</span> app = Server::with_context(handle); </span> app.</span>at</span>("</span>/submit</span>").</span>post</span>(handler); </span> </span> </span>// Start both the actor and the server </span> </span>// and execute them concurrently </span> </span>let</span> a = app.</span>listen</span>("</span>127.0.0.1:8080</span>"); </span> </span>let</span> b = actor.</span>run</span>(); </span> (a, b).</span>try_join</span>().await?; </span> </span> Ok(()) </span>} </span></code></pre> And now that our server has a handle, we can change our request handler function to schedule work on the actor rather than in a dangling task:</p> async </span>fn </span>handler</span>(</span>req</span>: Request, </span>handle</span>: Handle) -> Result<Response> { </span> </span>let</span> body = req.</span>body</span>().</span>try_to_string</span>()?; </span> handle.</span>send</span>(body).await?; </span>// ← Send work to the actor </span> Ok(Response::Ok()) </span>} </span></code></pre> While this still relies on performing an async operation that may potentially fail, this is still meaningfully different. Rather than waiting for the work to have been completed</em>, all we’re now waiting for is for the work to have been enqueued</em>. And unless something is going horribly wrong in the system, that should happen virtually instantly. And if for some reason it doesn’t: that’s why we always add timeouts to our computations right? …right?</a></p> How are actors different from globals?</h2> This section was added on 2025-06-02, after publishing the post.</em></p> I’ve had a number of people ask what the difference is between the actor pattern shown in this post, and the global task pool that manages the spawned tasks. After all: if you take the program as the root, both variants end up looking tree-shaped.</p> The difference between the two is that under structured concurrency we’re not treating the program</em> as the root, but the main function</em> as the root. The actor pattern in this post turns an unreachable, global task pool into a reachable task pool with a locality that can be managed however we like. Sy Brand put this particularly well; paraphrasing ever so slightly:</p> […] that "oh, it's structured as long as you look at it this way" smells like "well my program doesn't leak memory since it will all be reclaimed when it terminates".</p> </blockquote> The point of structured concurrency is that everything is always managed and reachable by having all computation existing in the same tree. This makes it possible to always</em> handle errors, restart instances, and shut things down gracefully.</p> Conclusion</h2> In this post we’ve answered one of the most common questions people have when first encountering structured concurrency: “How do I schedule background work without relying on dangling tasks?”</em></p> The simplest answer to this question are actors: types that provide some (shareable) handle that can be used to receive work on. We’ve shown how to define an actor in about 15 lines of code, how to integrate it with an HTTP server, and finally using it to replace an existing background task.</p> I think of structured concurrency the way I think about memory safety: the more we make it the default, the fewer concurrency issues people will have overall. And that seems important because concurrent, asynchronous, and parallel computing tends to some of the hardest code to reason about and debug.</p> Appendix A: Actor pattern template</h2> use </span>async_channel::{</span>self </span>as channel, Sender, Receiver}; </span>use </span>futures_concurrency::prelude::; </span> </span>type </span>Message = (); </span>type </span>Handle = Sender<Message>; </span> </span>struct </span>Actor(Receiver<Message>); </span>impl </span>Actor { </span> </span>fn </span>new</span>(</span>capacity</span>: </span>usize</span>) -> (</span>Self</span>, Handle) { </span> </span>let </span>(sender, receiver) = channel::bounded(capacity); </span> (</span>Self</span>(receiver), sender) </span> } </span> </span> async </span>fn </span>run</span>(&</span>mut </span>self</span>) -> Result<()> { </span> </span>self</span>.</span>0.</span>co</span>().</span>try_for_each</span>(\|</span>msg</span>\| { </span> todo!("</span>schedule work here</span>") </span> }).await </span> } </span>} </span></code></pre> Async Traits Can Be Directly Backed By Manual Future Impls Mon, 26 May 2025 00:00:00 +0000 Introduction</h2> There’s a fun little tidbit that most people don’t seem to be aware of when writing async functions in traits (AFITs) and that is: they allow you to directly return futures from methods. Sounds confusing? Let me illustrate what I mean with a simple example. Assume we have an a trait AsyncIterator</code> which has an async fn next</code> method that is defined as follows:</p> trait </span>AsyncIterator { </span> </span>type </span>Item; </span> async </span>fn </span>next</span>(&</span>mut </span>self</span>) -> Option<</span>Self::</span>Item>; </span>} </span></code></pre> If we wanted to define an iterator which yields an item exactly once, we would probably write it as follows. This stores a value T</code> in an Option</code>, and in a call to next</code> we extract it and return Some(T)</code> if we have a value, and None</code> if we don’t:</p> /// Yields an item exactly once </span>pub struct </span>Once<T>(Option<T>); </span>impl</span><T> AsyncIterator </span>for </span>Once<T> { </span> </span>type </span>Item = T; </span> async </span>fn </span>next</span>(&</span>mut </span>self</span>) -> Option<T> { </span> </span>self</span>.</span>0.</span>take</span>() </span> } </span>} </span></code></pre> Easy right? Now let’s try and write this future by hand. Rust is a systems programming language, and so it’s important that we can drop into lower levels of abstraction to gain additional control when needed. I consider it to be a gap in the language whenever we can’t break through an abstraction into its constituent parts. So let’s see what that would look like.</p> Naively backing AFITs with manual future impls</h2> Ok, so what are the constituent parts of an async fn</code>? Well, that’s mainly the Future</code> trait. What we want to do here is to define our own future and use that as the basis of our implementation. All we do is deref an option, and take its internal value - which makes it rather simple. Naively we would probably write something like this:</p> /// The internal `Future` impl of our `Once` iterator </span>struct </span>OnceFuture<</span>'a</span>, T>(&</span>'a mut </span>Once<T>); </span>impl</span><</span>'a</span>, T> Future </span>for </span>Once<</span>'a</span>, T> { </span> </span>type </span>Output = T; </span> </span>fn </span>poll</span>(</span>self</span>: Pin<&</span>mut Self</span>>, </span>cx</span>: &</span>mut </span>Context<</span>'a</span>>) -> Poll<</span>Self::</span>Output> { </span> </span>// SAFETY: we're projecting into an unpinned field </span> </span>let</span> this = </span>unsafe </span>{ Pin::into_inner_unchecked(</span>self</span>) }; </span> Poll::Ready((&</span>mut</span> this.</span>0.0</span>).</span>take</span>()) </span> } </span>} </span> </span>/// Yields an item exactly once </span>pub struct </span>Once<T>(Option<T>); </span>impl</span><T> AsyncIterator </span>for </span>Once<T> { </span> </span>type </span>Item = T; </span> async </span>fn </span>next</span>(&</span>mut </span>self</span>) -> Option<T> { </span> </span>// Delegate to the `OnceFuture` impl </span> OnceFuture(</span>self</span>).await </span> } </span>} </span></code></pre> There’s a lot going on all of a sudden, right? This is the low-level version of what we wrote earlier. Or is it? If you look carefully you might notice that we’re dealing with one extra level of indirection here. In our iterator’s main body we wrote: OnceFuture(self).await</code>. In a simple benchmark the compiler will happily optimize this. But in more complex programs the compiler may not. And that’s an issue, because it means that switching to a lower-level abstraction may yield worse performance.</p> If this was the best we could do, it would likely spell the death of AFITs in the stdlib. It would mean that AFITs would be useful for high-level APIs that we can implement using async fn</code> and be happy with that. But not for serious</em> implementations which need to be optimized all the way, like those found in the stdlib. And that would point us at things like poll_</code>-based traits which we can more directly implement by hand.</p> Directly backing AFITs with manual future impls</h2> Luckily we can trivially remove the intermediate .await</code> call from our previous example using a little-known AFIT feature: the ability to directly return futures. Instead of writing async fn next</code>, we can write that same function as fn next() -> impl Future</code>. That’s almost the same; with some minor differences:</p> /// The internal `Future` impl of our `Once` iterator </span>struct </span>OnceFuture<</span>'a</span>, T>(&</span>'a mut </span>Once<T>); </span>impl</span><</span>'a</span>, T> Future </span>for </span>Once<</span>'a</span>, T> { </span> </span>type </span>Output = T; </span> </span>fn </span>poll</span>(</span>self</span>: Pin<&</span>mut Self</span>>, </span>cx</span>: &</span>mut </span>Context<</span>'a</span>>) -> Poll<</span>Self::</span>Output> { </span> </span>// SAFETY: we're projecting into an unpinned field </span> </span>let</span> this = </span>unsafe </span>{ Pin::into_inner_unchecked(</span>self</span>) }; </span> Poll::Ready((&</span>mut</span> this.</span>0.0</span>).</span>take</span>()) </span> } </span>} </span> </span>/// Yields an item exactly once </span>pub struct </span>Once<T>(Option<T>); </span>impl</span><T> AsyncIterator </span>for </span>Once<T> { </span> </span>type </span>Item = T; </span> </span>// Directly returns the `OnceFuture` impl </span> </span>fn </span>next</span>(&</span>mut </span>self</span>) -> impl Future<Output = Option<T>> { </span> OnceFuture(</span>self</span>) </span>// ← no more `.await` </span> } </span>} </span></code></pre> Do you see the fn next</code> impl? It now directly returns the OnceFuture</code>. And we can do this even though the trait definition explicitly defined the trait method as async fn next</code>. This means there is no intermediate .await</code> call required to base our AFIT-based trait on a manual future impl. Which means we’re no longer depending on the compiler to optimize this for us to have equivalent performance, meeting our goals of being able to manually lower the high-level notation into its constituent parts which we can manually control.</p> Simple inline poll state machines</h2> The Rust stdlib provides a future::poll_fn</code> convenience function that allows you to create stateless (e.g. does not hold its own state across poll</code> calls) futures. But because it takes an FnMut</code> closure, it can reference external state made available to it via upvars. This allows us to rewrite the chonky manual future implementation from our last two examples as a sleek inline call to poll_fn</code>:</p> /// Yields an item exactly once </span>pub struct </span>Once<T>(Option<T>); </span>impl</span><T> AsyncIterator </span>for </span>Once<T> { </span> </span>type </span>Item = T; </span> </span>fn </span>next</span>(&</span>mut </span>self</span>) -> impl Future<Output = Option<T>> { </span> future::poll_fn(\|</span>_cx</span>\| </span>/ -> Poll<Option<T>> / </span>{ </span> Poll::Ready((&</span>mut </span>self</span>.value).</span>take</span>()) </span> }) </span> } </span>} </span></code></pre> This is incredibly convenient because it allows us to quickly write some optimized inline poll-based state machine code inside trait implementations. Or really: any async</code> context. I like to think of poll_fn</code> as a way to quickly step into “low level async” mode, kind of like how unsafe {}</code> allows you to step into “raw memory mode”.</p> This also clearly makes the Future</code> trait the fundamental building block of everything async</code>. Which is a simpler and more robust model than the alternative fn poll_</code> model; under which async operations can be implemented either in terms of Future::poll</code>, AsyncIterator::poll_next</code>, AsyncRead::poll_read</code>, and so on.</p> Conclusion</h2> I wrote this post because most people don’t seem to be aware that AFITs can be used like this. Though arguably it’s one of its most important capabilities. And that’s important because as I said earlier in this post: it would be really bad if the async trait space was bifurcated between:</p> AFIT-based traits</strong>: which are convenient to implement but have worse performance due to lack of control.</li> Poll-based traits</strong>: which are inconvenient to implement but have better performance because of the control provided</li> </ul> This post shows that AFIT-based traits are both convenient to implement, and when desired provide the control needed to guarantee performance via manual poll-state machine implementations.</strong> This unifies the design space for async traits, removing the choice between “the easy API” and “the fast API”. With AFITs the easy API is</em> the fast API.</p> While we used the AsyncIterator</code> trait as an example in this post, nothing in this post so far has been specific to AsyncIterator</code>. Being able to control the future state machines of AsyncRead</code>, AsyncWrite</code>, and so on isn’t any less important. But if we were to consider an AFIT-based AsyncIterator</code> specifically with an async gen {}</code> feature we can see that it provides three total levels of control, where a poll</code>-based AsyncIterator</code> trait only provides two ^{1</a></sup>:</p>} </th> AFIT-based</th> poll_</code>-based</th></tr></thead> async gen {}</code></td> ✅</td> ✅</td></tr> async fn</code></td> ✅</td> ❌</td></tr> fn poll</code> state machine</td> ✅</td> ✅</td></tr> </tbody></table> And all three levels of abstraction are important to be convenient to implement. It would be wrong to pick and choose. Being able to implement async traits based on async fn</code> is important in cases where a high-level construct like gen {}</code> is not available, like with AsyncRead</code>, and AsyncWrite</code>. But it’s also important for AsyncIterator</code> to be able to implement subtrait variants like ExactSizeIterator</code>, and be able to implement provided methods like fn size_hint</code>. async gen {}</code> doesn’t provide the ability to do either ^{2</a></sup>, and users shouldn’t be forced to write poll</code>-based state machines just for that.</p>} Acknowledgements</h2> I’d like to thank Oli Scherer for taking the time late last year to work with me through a series of examples of AFITs backed manually implemented futures, and explaining how they are resolved and optimized by the compiler. That conversation is what has given me the confidence to make the claims in this post with a degree of certainty I would otherwise not have. Oli did however not review this post in advance, so I’m most definitely not speaking on their behalf and any mistakes in this post will be my own.</p> ^{1</sup> I’m sure with enough language magic we could provide some</em> way to allow fn poll_</code>-based traits to be implemented in terms of async fn</code>. But that require an entirely new language feature, all to just end up with a less consistent and more complicated model than if we directly base all async traits (but Future</code>) on async fn</code>.</p> </div>}^{2</sup> Capability-based arguments aside, I also believe that implementing traits should</em> be easy. Like, what do you mean implementing a non-blocking iterator by hand requires a PhD in Rustology? Rust is supposed to make systems programming accessible and understandable</em>. And that needs to apply to every level of abstraction.</p> </div>} The Waker Allocation Problem Fri, 23 May 2025 00:00:00 +0000 Introduction</h2> The entire point of futures and Rust’s async/.await system is to introduce two new capabilities: arbitrary concurrent execution of computations, and arbitrary cancellation of computations. The difference between blocking and non-blocking computation is unimportant if we then don’t also capitalize on it by scheduling work concurrently.</p> But there’s a problem - In order to schedule work concurrently futures have two bad choices in how to organize their internals:</p> Without allocations</strong> futures have O(N^2)</code>(quadratic) wakeup behavior for their child futures. This means that if we schedule 1_000 futures that all wake independently and in sequence, we’ll approximate 1_000_000 total wakes 1</a></sup>.</li> With allocations</strong> futures can have O(N)</code> wakeup behavior for their child futures. This means that if we schedule 1_000 futures that all complete independently and in sequence, we’ll have 1_000 total wakes.</li> </ul> In Rust we’re well aware that hidden allocations are bad. But we also know that quadratic scaling on potentially unbounded numbers of items is also bad. And it’s bad that we’re in a position where we have to choose between both.</p> Efficient concurrent execution requires intermediate wakers</h2> Say we’re writing a manual future::join</code> function which takes an array of impl Future</code> and returns an array of Future::Output</code>. We can model the internal state of this using an array of futures, and an array of future outputs:</p> /// A future that concurrently awaits a list of futures </span>struct </span>Join<Fut: Future, </span>const</span> N: </span>usize</span>> { </span> </span>/// The futures we're awaiting. </span> </span>futures</span>: [Fut; N], </span> </span>/// The outputs of the futures. </span> </span>outputs</span>: [Option<</span>Fut::</span>Output>; N], </span>} </span></code></pre> In order to concurrently await all futures, we have to implement the Future</code> trait for Join</code>. The output type of this is [Future::Output; N]</code>, with each call to poll</code> iterating over each pending future contained within. Here’s an naive example implementation that, mostly just to provide context:</p> impl</span><Fut: Future, </span>const</span> N: </span>usize</span>> Future </span>for </span>Join<Fut, N> { </span> </span>type </span>Output = [Fut::Output; N]; </span> </span> </span>fn </span>poll</span>(</span>self</span>: Pin<&</span>mut Self</span>>, </span>cx</span>: &</span>mut </span>Context<'_>) -> Poll<</span>Self::</span>Output> { </span> </span>// setup the initial state </span> </span>let</span> this = </span>unsafe </span>{ </span>self</span>.</span>get_unchecked_mut</span>() }; </span> </span>let mut</span> all_done = </span>true</span>; </span> </span> </span>// Iterate over every future until we either have all </span> </span>// values, or we still have futures pending. </span> </span>for</span> i in </span>0</span>..this.futures.</span>len</span>() { </span> </span>// Either the future is already done… </span> </span>if</span> this.outputs[i].</span>is_some</span>() { </span> </span>continue</span>; </span> } </span> </span> </span>// …or we still need to poll it. </span> </span>let</span> fut = </span>unsafe </span>{ Pin::new_unchecked(&</span>mut</span> this.futures[i]) }; </span> </span>match</span> fut.</span>poll</span>(cx) { </span> Poll::Ready(output) => this.outputs[i] = Some(output), </span> Poll::Pending => all_done = </span>false</span>, </span> } </span> } </span> </span> </span>// The loop is done and its time to look at our </span> </span>// results. We either still have futures pending… </span> </span>if </span>!all_done { </span> </span>return </span>Poll::Pending; </span> } </span> </span> </span>// …or we're all done and we're ready to return. </span> </span>let</span> outputs = std::mem::replace(&</span>mut</span> this.outputs, [None; N]) </span> .</span>map</span>(Option::unwrap); </span> </span> Poll::Ready(outputs) </span> } </span>} </span></code></pre> If you look at the core of this loop, you’ll notice that we’re iterating over each future on every iteration - regardless of whether it called its respective waker of not. The only futures we don't call are the ones we already have an output of. Practically this means that we approach O(N^2)</code> execution: we wake every future on every call to poll. And we do this because we don’t know which futures were are ready and which are not.</p> The solution here is to create an “intermediate” or “inline” waker. Rather than passing the caller’s waker down into all of the child futures, the Join</code> future should instead create its own waker(s) which can track which child futures are ready - and only poll those. The simplest way of doing this is keeping an array of wakers and an array of “ready” indexes. Whenever a child future calls wake, it pushes its index to the list of ready indexes, and then calls the parent waker. Then when the Join</code> future is polled, all it needs to do is iterate over all ready indexes and poll those futures. For brevity I won’t bother showing the full implementation, just the definition itself:</p> /// A future that concurrently awaits a list of futures </span>struct </span>Join<Fut: Future, </span>const</span> N: </span>usize</span>> { </span> </span>/// The futures we're awaiting. </span> </span>futures</span>: [Fut; N], </span> </span>/// The outputs of the futures. </span> </span>outputs</span>: [Option<</span>Fut::</span>Output>; N], </span> </span>/// Keeps a waker per child future </span> </span>wakers</span>: [ChildWaker; N], </span> </span>/// Tracks which child futures should be woken </span> </span>indexes</span>: Arc<[AtomicBool; </span>N</span>]>, </span>} </span> </span>/// A waker associated with a specific child future </span>struct </span>ChildWaker { </span> </span>/// The index of the child future in the index list </span> </span>index</span>: </span>usize</span>, </span> </span>/// A handle to the parent waker, passed to the struct </span> </span>parent_waker</span>: Waker, </span> </span>/// Tracks which child futures should be woken. Intended </span> </span>/// to be indexed into using `index` inside `Wake`. </span> </span>indexes</span>: Arc<[AtomicBool; </span>N</span>]>, </span>} </span>impl </span>Wake </span>for </span>InlineWaker { ... } </span></code></pre> There are more efficient schemes than this one, though this is among the simplest. And crucially: this allows us to only ever wake the futures that need to be woken. Which is important, since waking futures may potentially be expensive. And while we can encourage futures to be cheap to wake spuriously - we should really just not cause spurious wakes in the first place. Because it's better if the burden falls on async experts writing primitives, than on every single person implementing the future trait 2</a></sup>.</p> Intermediate wakers require allocations</h2> Now that we’ve looked at why futures want to make use of intermediate wakers, let’s take a closer look at the waker internals. The Future</code> trait relies on a type called Waker</code>, which you can think of as an Arc<Fn></code> pointer. Where you can think of Arc</code> as a fancy type of Box<T></code> that can be freely cloned around and provides shared (immutable) access to its internals. Here’s how you define a ew Waker</code> that will unpark the current thread when called:</p> use </span>std::sync::Arc; </span>use </span>std::task::{Context, Wake}; </span>use </span>std::thread::{</span>self</span>, Thread}; </span>use </span>std::pin::pin; </span> </span>/// A waker that wakes up the current thread when called. </span>struct </span>ThreadWaker(Thread); </span> </span>impl </span>Wake </span>for </span>ThreadWaker { </span> </span>fn </span>wake</span>(</span>self</span>: Arc<</span>Self</span>>) { </span> </span>self</span>.</span>0.</span>unpark</span>(); </span> } </span>} </span></code></pre> And here is how you construct an instance of that same waker. This relies on a From<Arc<Wake + Send + Sync + 'static>></code> impl</a> for `Waker:</p> /// Run a future to completion on the current thread. </span>fn </span>block_on</span><T>(</span>fut</span>: impl Future<Output = T>) -> T { </span> </span>let mut</span> fut = pin!(fut); </span> </span> </span>// Construct an instance of `ThreadWaker` </span> </span>let</span> t = thread::current(); </span> </span>let</span> waker = Arc::new(ThreadWaker(t)).</span>into</span>(); </span>// ← This allocates </span> </span> </span>// ... </span></code></pre> Right there, creating an instance of Arc::new</code> means we’re allocating. In theory we don’t have</em> to use Arc</code> for this either. The type Waker</code></a> only requires that it wraps a RawWakerVTable</code></a> in a way that is Send + Sync + Clone</code>. That means in theory we could also have a static</code> somewhere with internal mutability, and perhaps some (embedded) systems rely on this for their global runtimes. But in practice for inline concurrent operations, if we’re going to be creating a Waker</code> it’s going to need to be based on an Arc</code>.</p> Conclusion</h2> The fact that it practically relies on allocations is not the only problem: in single-threaded systems Waker</code> still requires the + Send + Sync</code> bounds. Even if we would end up adding a LocalWaker</code></a> type, that still requires the original Waker</code> type to be present. Which means we can’t practically write portable async-based !Send</code> libraries, because that bound will never surface in the type system 3</a></sup>.</p> The reason why we have an Arc<Fn></code> to begin with is because the future design assumes we might want to wrap arbitrary computation in the wakers. But I’m not sure how true that is. For example, in the futures-concurrency</code> library - our wakers only ever perform an operation equivalent to flipping an atomic bool (src</a>). If we were hand-rolling our own implementation directly against e.g. the epoll</code> API, we wouldn’t have to pay any cost like this. What we’ve looked at in this post is purely an artifact of the Future</code> trait’s design.</p> Having to choose between algorithmic complexity and allocations is also the reason why I haven't proposed to upstream futures-concurrency APIs to the stdlib yet. The Rust stdlib has a design principle that disallows hidden allocations. But at work we also know we need to guard against API misuse. That means we’re likely going to hit cases where using the stdlib APIs are actually not</em> recommended. And it’s hard to justify upstreaming something we might immediately want to guard against using.</p> This is also not the only problem with the Future</code> trait design. I’ve spoken at length about the problems the related Pin</code> API has. And in this post I've also touched on the !Send</code> Waker problem. By my count the Future</code> trait has seven problems like this, which I’ll try to describe in future posts.</p> ^{2</sup> This is not just a hypothetical - its an issue we’ve had at Microsoft. We can’t reasonably expect that every person implementing the future trait is an async expert. And so we need primitives which don’t just pass responsibility for performance to individual users.</p> </div> ^{1</sup> To be fair - probably about half that, since futures which have completed aren’t polled again.</p> </div> ^{3</sup> I hope I don’t have to explain that having Send</code> expressed in the type system is a good thing.</p> </div>}}} Open and Closed Effect Systems Thu, 22 May 2025 00:00:00 +0000 When discussing effect systems in the context of production programming languages, I believe that the most important distinction is whether effects are purely defined by the language, or whether users are also able to define them. When Magnus Madsen from the Flix programming language</a> joined the Rust T-Lang call for two sessions last December, he used the terms “open” and “closed” to describe this distinction:</p> Closed effect systems</strong>: are systems where all effects are all defined by the language as built-ins. The number of effects may grow over time, but effects are only ever defined by the language itself. This makes the number of possible effects bounded</em>.</li> Open effect systems</strong>: allow users of the language to also define their own control-flow effects, which exist alongside the language built-in effects. This makes the number of possible effects unbounded</em>.</li> </ul> The most interesting effects in programming languages have</em> to be built-in. This includes things like controlling whether a function guarantees it will terminate (“divergence”). Or whether a function is allowed to perform side effects (“purity”).</p> This is different from user-defined effects. These are typically defined using “algebraic effect handlers”, but I’ve also seen the term “typed continuation” used 1</a></sup>. I think of these more or less as language-level support for thoroughly typed iterators/generators/coroutines. It doesn’t really make sense to express “divergence” using iterators. Which is why even though all effect handlers are effects, not all effects are expressed using effect handlers.</p> In the context of Rust all we’re discussing right now is the introduction of a closed</em> effect system. And that’s not because we don’t think an open system would not be useful or good. But because formalizing and generalizing Rust’s effect system in a backwards-compatible way is an incredible amount of work already. And in order to ship, we have to draw the line somewhere 2</a></sup>.</p> ^{1</sup> I believe there is a difference between the two terms, but I don’t quite remember. I find “typed continuation” to be a lot more evocative though. Coming from Node.js 0.8, I think of this kind of as “what if all callbacks were fully typed and we had a compiler that ensured they were handled”. Maybe not syntactically the same, but I think it’s semantically quite close.</p> </div> ^{2</sup> Though that doesn’t mean we can’t keep syntactic space open to consider it later. Just because we’re saying “not now” doesn’t mean we’re saying “not ever”.</p> </div>}} Automatic interleaving of high-level concurrent operations Mon, 05 May 2025 00:00:00 +0000 Introduction</h2> When working with low-level concurrency (atomics), programming languages are generally quite eager for compilers to reorder operations if it leads to better performance. Information about whether it's ok to reorder operations is encoded using Atomic Orderings</a>, Fences</a>, and Operations</a>. It's strange that most programming languages that support semantic-aware reordering of low-level concurrent operations, don't also include similar support for reordering the execution of high-level concurrent operations.</p> To my knowledge this is true for most languages, with the notable exception of Swift and its async let</code> construct. This feature preserves the linear-looking nature of async code, but allows the compiler to inspect the control flow graph and schedule operations concurrently where possible. That means that just like with atomics, an operation that is defined later in a piece of code may finish execution before an operation that appears earlier. Here is an example Swift program where everything that can</em> be concurrent actually is</em> concurrent:</p> func </span>makeDinner() async throws -> Meal { </span> async </span>let</span> veggies = chopVegetables() </span>// 1. concurrent with: 2, 3 </span> async </span>let</span> tofu = marinateTofu() </span>// 2. concurrent with: 1, 3 </span> async </span>let</span> oven = preheatOven(temperature: </span>350</span>) </span>// 3. concurrent with: 1, 2, 4 </span> </span> </span>let</span> dish = Dish(ingredients: await [</span>try</span> veggies, tofu]) </span>// 4. depends on: 1, 2, concurrent with: 3 </span> </span>return</span> await oven.cook(dish, duration: .hours(</span>3</span>)) </span>// 5. depends on: 3, 4, not concurrent </span>} </span></code></pre> The Problem</h2> To me this represents the pinnacle of language-level support for asynchronous/concurrent programming. It makes it trivial to change any code that may</em> be run concurrently to actually run concurrently. It enables the compiler to take care of what is otherwise tedious and/or illegible. Take for example this code that's written in a serial fashion using async/.await:</p> async </span>fn </span>make_dinner</span>() -> SomeResult<Meal> { </span> </span>let</span> veggies = </span>chop_vegetables</span>().await?; </span> </span>let</span> tofu = </span>marinate_tofu</span>().await?; </span> </span>let</span> oven = </span>preheat_oven</span>(</span>350</span>).await; </span> </span> </span>let</span> dish = Dish(&[veggies, tofu]).await; </span> oven.</span>cook</span>(dish, Duration::from_mins(</span>3 </span>* </span>60</span>)).await </span>} </span></code></pre> Using Future::join</code> operations we can rewrite it to execute independent operations concurrently. But this comes with the downside that the code is now significantly less legible. Here is the same code, written using Future::try_join</code>:</p> use </span>futures_concurrency::prelude::; </span> </span>async </span>fn </span>make_dinner</span>() -> SomeResult<Meal> { </span> </span>let</span> dish_fut = { </span> </span>let</span> veggies_fut = </span>chop_vegetables</span>(); </span> </span>let</span> tofu_fut = </span>marinate_tofu</span>(); </span> </span>let </span>(veggies, tofu) = (veggies_fut, tofu_fut).</span>try_join</span>().await?; </span> Dish::new(&[veggies, tofu]).await </span> }; </span> </span>let</span> oven_fut = </span>preheat_oven</span>(</span>350</span>); </span> </span>let </span>(dish, oven) = (dish_fut, oven_fut).</span>try_join</span>().await?; </span> oven.</span>cook</span>(dish, Duration::from_mins(</span>3 </span> </span>60</span>)).await </span>} </span></code></pre> To capitalize on one of the core features of async/.await</code> (ad-hoc concurrent scheduling</em>), we had to sacrifice one of its main benefits (legibility).</strong> It's not good when two core parts of the same feature are in tension with each other like that. And we can't just wave a wand and tell the compiler to automatically execute these futures concurrently. Futures tend to express operations that change program state in one way or another. That is to say: most futures encode side-effects</em>. And the compiler can't automatically infer which side effects can be executed serially and which can be executed concurrently. That's because it's not aware of the program semantics</em>.</p> The Solution</h2> The solution is to allow programmers to provide opt-in to explicit reorderings in their code, just like Swift does using async let</code>. We could use a concise notation along the lines of .co.await</code> (this is a strawman, pick your own favorite notation). We want the notation to be in postfix position because unlike Swift we don't want to eagerly start executing when operations are defined, but only affect the way operations are scheduled when .await</code>ed. And this way we never have to actually have to represent it in the type system either 1</a></sup>. This would look something like this:</p> ^{1</sup> Writing .co</code> without following it with an .await</code> should be a compiler error. The .co</code> would serve as a modifier on the .await</code>. Though perhaps something like C++'s co_await</code> notation is simpler. Whatever the syntax though, I don't think it should ever surface to the type system.</p> </div>}async </span>fn </span>make_dinner</span>() -> SomeResult<Meal> { </span> </span>let</span> veggies = </span>chop_vegetables</span>().co.await?; </span> </span>let</span> tofu = </span>marinate_tofu</span>().co.await?; </span> </span>let</span> oven = </span>preheat_oven</span>(</span>350</span>).co.await; </span> </span> </span>let</span> dish = Dish(&[veggies, tofu]).co.await; </span> oven.</span>cook</span>(dish, Duration::from_mins(</span>3 </span>* </span>60</span>)).await </span>} </span></code></pre> This code would directly lower to the equivalent of the Future::join</code>-based scheme. But with the benefit of needing far less ceremony to encode the same semantics. The other benefit of this scheme is that we always retain the option to schedule these operations serially if we choose to. That makes this scheme compatible with async-polymorphic functions, where manual Future::join</code> calls are not.</p> A feature along these lines is important, because in order to make full use of async Rust's concurrent scheduling capabilities, any operations which can be executed concurrently should be executed concurrently. But without language support that comes at a severe cost to legibility, and in turn maintainability. The only way out of this dilemma is to do what Swift has done and directly include language support for it.</p> Further Reading</h2> Tasks are the Wrong Abstraction</a> - Introduces the ParallelFuture</code> trait</a>, which can be combined with the scheme outlined in this post to automatically schedule futures on multiple cores.</li> Tree-Structured Concurrency</a> - Discusses what structured concurrency is, how to reason about it, and what's missing for Rust to fully support it.</li> Extending Rust's Effect System</a> - Discusses among other things async-polymorphic functions.</li> </ul> Syntactic musings on match expressions Tue, 29 Apr 2025 00:00:00 +0000 Introduction</h2> One of the things that stands out to me is how similar match</code> and if..else</code> are semantically</em>, while being diverging a fair bit syntactically</em>. The reasons for that seem mostly accidental, and I have a sneaking suspicion we could make things a little easier by making both constructs look more similar.</p> In this post I'll be showing some control flow examples based on a queue that can either be online of offline. This is defined by an enum QueueState</code> which stores the length of the queue as a u32:</p> enum </span>QueueState { </span> Online(</span>u32</span>), </span> Offline(</span>u32</span>), </span>} </span></code></pre> Depending on whether the queue is actively having new data added to it (online), we will want to reason differently about the queue's limits (u32</code>). We also want to know change the behavior based on whether the queue is full or not, and in order to do that we will want to call a function:</p> fn </span>is_full</span>(</span>n</span>: </span>u32</span>) -> </span>bool </span>{ .. } </span></code></pre> Logical AND</h2> Take the humble if</code>-guard. We can use this in match</code> arms to chain conditions, which is useful when we want to do something with a value contained within a structure. Here the is_full</code> method returns a bool. But in order to call it we have first access the number contained in the Online</code> variant:</p> match</span> state { </span> QueueState::Online(count) </span>if </span>!</span>is_full</span>(count) => { </span> </span>// ^^^^^^^^^^^^^^^^^^ </span> </span>// if-guard </span> println!("</span>queue is online and not full</span>"); </span> } </span> _ => println!("</span>queue is in an unknown state</span>"), </span>} </span></code></pre> if</code>-guards are really just a logical AND. In match statements we're in the process of adding if let</code>-chaining to Rust by way of RFC 2497</a> (stable in Rust 1.88). With that we'll be able to write this same match</code> condition as follows:</p> if let </span>QueueState::Online(count) = state && !</span>is_full</span>(count) { </span>// ^^^^^^^^^^^^^^^^^^ </span>// && chaining </span> println!("</span>queue is online and not full</span>"); </span>} </span>else </span>{ </span> println!("</span>queue is in an unknown state</span>"); </span>} </span></code></pre> This is basically the same feature exposed two different ways. Instead I really would like to be able to use &&</code> in both cases, bringing the two closer together:</p> match</span> state { </span> QueueState::Online(count) && !</span>is_full</span>(count) => { </span> </span>// ^^^^^^^^^^^^^^^^^^ </span> </span>// && chaining </span> println!("</span>queue is online and not full</span>"); </span> } </span> _ => println!("</span>queue is in an unknown state</span>"), </span>} </span></code></pre> I can guarantee that just about everyone who has had exposure to a language which C-like notation (which rounds to all programmers) will have little trouble understanding how this works. if</code>-guards, in comparison, tend to be harder to intuit.</p> Logical OR</h2> RFC 3637</a> introduces the "guard patterns" feature: an extension to Rust's pattern notation which that moves if</code> guards out of match expressions and into all patterns. What it functionally provides us with is a way to chain boolean AND expressions with logical OR expressions in all patterns. Say we took our previous example, and we wanted to print a message if is_full</code> returned false</code>, regardless of the whether the queue was online or offline. Using chained if..let</code>s we would write this as follows:</p> if let </span>QueueState::Online(count) = state && !</span>is_full</span>(count) { </span> println!("</span>queue is online and not full</span>"); </span>} </span>else if let </span>QueueState::Online(count) = state && </span>is_full</span>(count) { </span> println!("</span>queue is full</span>"); </span>// 1. This statement... </span>} </span>else if let </span>QueueState::Offline(count) = state && </span>is_full</span>(count) { </span> println!("</span>queue is full</span>"); </span>// 2. ...is duplicated here. </span>} </span>else </span>{ </span> println!("</span>queue is in an unknown state</span>"); </span>} </span></code></pre> This is a little annoying because ideally we'd want both the second and third condition to be one big condition, separated by a logical OR</code>. Guard patterns are the feature which will allow us to do exactly that by bringing if</code>-guards and pattern OR-based pattern chaining to expressions. With that we would rewrite this as follows:</p> if let </span>QueueState::Online(count) </span>if </span>!</span>is_full</span>(count) = state { </span> </span>// ^^^^^^^^^^^^^^^^^^ </span> </span>// now using an if-guard, for fun </span> println!("</span>queue is online and not full</span>"); </span>} </span>else if let </span>((QueueState::Online(count) </span>if </span>is_full</span>(count) = state) \| </span>// logical OR </span> </span>// ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ </span> </span>// Either matches on pattern 1... </span> (QueueState::Offline(count) </span>if </span>is_full</span>(count) = state) </span> </span>// ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ </span> </span>// ...or on pattern 2. </span>{ </span> println!("</span>queue is full</span>"); </span>// No more duplication! </span>} </span>else </span>{ </span> println!("</span>queue is in an unknown state</span>"); </span>} </span></code></pre> There's a lot going on here. For one, this makes if</code>-guards legal in if..let</code> expressions. And it also allows multiple patterns to be placed used in a single conditional using OR semantics, as long as they're appropriately parenthesized. Let's keep going on, and use this with a match</code> expression:</p> match</span> state { </span> QueueState::Online(count) </span>if </span>!</span>is_full</span>(count) => { </span> println!("</span>queue is online and not full</span>"); </span> } </span> ((QueueState::Online(count) </span>if </span>is_full</span>(count)) \| </span> (QueueState::Offline(count) </span>if </span>is_full</span>(count)) => { </span> println!("</span>queue is full</span>"); </span>// No more duplication! </span> } </span> _ => println!("</span>queue is in an unknown state</span>"), </span>} </span></code></pre> This is a lot better since the flow is clearer. By not having the explicit = state</code> repeated on every line we can now actually read all the statements left-to-right. That's all good. We like that. Porting from the if..let</code> version to the match</code> version was also really easy. We didn't have to change the if..let</code> conditions at all, aside from trimming their ends. That's good!</p> What's less good however is that we achieved this by making if..else</code> expressions more involved. The interaction of RFC 2497 if..let</code> chains and RFC 3637 guard patterns seems particularly like it might catch people off guard. Consider for example this line:</p> if let </span>Foo::Bar(x) </span>if </span>cond</span>(x) = </span>bar</span>() && </span>let </span>Bin::Baz = </span>baz</span>() { .. } </span></code></pre> The order in which expressions are evaluated on this line is here is 2-3-1-5-4</code>. That means it's probably not advisable to mix both features. The reason why these interactions are this way is because both RFCs operate from fundamentally opposing premises:</strong></p> RFC 2497</strong> believes we should enable boolean operators like &&</code> to work in more places.</li> RFC 3637</strong> believes we should enable match-specific operators like if</code>-guards work in more places.</li> </ul> In the previous section we already looked at replacing if</code>-guards with the &&</code> operator, inspired by if..let</code>-chaining. It's not a far reach to imagine if..let</code>-chaining using the \|\|</code> operator as well. That would look something like this:</p> if let </span>QueueState::Online(count) = state && !</span>is_full</span>(count) { </span> </span>// ^^^^^^^^^^^^^^^^^^ </span> </span>// if-let chain </span> println!("</span>queue is online and not full</span>"); </span>} </span>else if let </span>QueueState::Online(count) = state && </span>is_full</span>(count) \|\| </span> </span>// ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ </span> </span>// Either matches on pattern 1... </span> </span>let </span>QueueState::Offline(count) = state && </span>is_full</span>(count) </span> </span>// ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ </span> </span>// ...or on pattern 2. </span>{ </span> println!("</span>queue is full</span>"); </span>// No more duplication! </span>} </span>else </span>{ </span> println!("</span>queue is in an unknown state</span>"); </span>} </span></code></pre> It's the same functionality as before, but provided as a natural extension to &&</code>-based if..let</code>-chaining. Now once more, let's try rewriting this as a match</code> expression:</p> match</span> state { </span> QueueState::Online(count) && !</span>is_full</span>(count) => { </span> println!("</span>queue is online and not full</span>"); </span> } </span> QueueState::Online(count) && </span>is_full</span>(count) \|\| </span>// <- logical OR! </span> QueueState::Offline(count) && </span>is_full</span>(count) => { </span> println!("</span>queue is full</span>"); </span> } </span> _ => println!("</span>queue is in an unknown state</span>"), </span>} </span></code></pre> What's neat here is that because logical AND (&&</code>) takes precedence over logical OR (\|\|</code>) (ref</a>), we don't need to resort to wrapping our patterns in order to chain them. Oh and I guess of course it's also nice that just about every programmer regardless of familiarity with Rust should be able to parse what is going on here.</p> But even more than that: look at this! This is code that I want to be writing. It's not trying to be clever or inventing anything particular new. It's defining feature is just how ordinary this looks. And that really agrees with my sensibilities for what a general-purpose programming language should feel like.</p> if let</code>-evaluation ordering</h2> I don't like reading if-let</code> statements because they flip the read order from right-to-left. I double don't like reading if-let</code> statements when chained because they need to be read center-left-right. I like the is</code>-operator because it allows us to fix that. Let me show you how by taking our last if let..else</code> example, with all comments stripped:</p> if let </span>QueueState::Online(count) = state && !</span>is_full</span>(count) { </span> println!("</span>queue is online and not full</span>"); </span>} </span>else if let </span>QueueState::Online(count) = state && </span>is_full</span>(count) \|\| </span> </span>let </span>QueueState::Offline(count) = state && </span>is_full</span>(count) </span>{ </span> println!("</span>queue is full</span>"); </span>} </span>else </span>{ </span> println!("</span>queue is in an unknown state</span>"); </span>} </span></code></pre> However much I like the boolean operators here, I don't like the order in which statements evaluate. The first line in our example above is evaluated as follows:</p> if let </span>QueueState::Online(cond) = state && !</span>is_full</span>(count) { .. } </span>// ^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^ ^^^^^^^^^^^^^^^ </span>// \| \| \| </span>// \| this is evaluated first \| </span>// \| \| </span>// this is evaluated second \| </span>// \| </span>// this is evaluated last </span></code></pre> RFC 3573</a> proposes to add the is</code>-operator to the language, inspired by C#'s feature of the same name. This would fix the evaluation order, making the same code instead read from top-to-bottom, right-to-left:</p> if</span> state is QueueState::Online(count) && !</span>is_full</span>(count) { </span> println!("</span>queue is online and not full</span>"); </span>} </span>else if</span> state is QueueState::Online(count) && </span>is_full</span>(count) \|\| </span> state is QueueState::Offline(count) && </span>is_full</span>(count) </span>{ </span> println!("</span>queue is full</span>"); </span>} </span>else </span>{ </span> println!("</span>queue is in an unknown state</span>"); </span>} </span></code></pre> To me this is Rust as it should be. The point has always been to make a language that is as practical as possible to use, without compromising on correctness, performance, and control. Making evaluation orders easier to follow seems like it's right in line with that goal. Here is that same line annotated, but using the is</code> notation:</p> if</span> state is QueueState::Online(cond) && !</span>is_full</span>(count) { .. } </span>// ^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^ </span>// \| \| \| </span>// this is evaluated first \| \| </span>// \| \| </span>// this is evaluated second \| </span>// \| </span>// this is evaluated last </span></code></pre> It also comes with an added benefit: it makes it so refactoring code from if..else</code> form to match</code> form produces an incredibly small diff. It's mostly the same code, but with the start of each condition trimmed:</p> match</span> state { </span> QueueState::Online(count) && !</span>is_full</span>(count) => { </span> println!("</span>queue is online and not full</span>"); </span> } </span> QueueState::Online(count) && </span>is_full</span>(count) \|\| </span> QueueState::Offline(count) && </span>is_full</span>(count) => { </span> println!("</span>queue is full</span>"); </span> } </span> _ => println!("</span>queue is in an unknown state</span>"), </span>} </span></code></pre> This would make it possible to explain conditional control flow more incrementally. I see the potential for carving out a little didactic staircase into Rust's famously steep learning curve that goes a little something like this:</p> Start by introducing if..else</code>, which most programmers will already know.</li> Then introduce is</code> as a more powerful version of typeof</code> in other languages.</li> Finally introduce match</code> as a more ergonomic version of if..else</code> + is</code>. Like switch</code> in other languages, but exhaustively checking all cases.</li> </ol> We never take any concepts away or have big changes in notation. Each concept neatly builds on the previous one, and we don't need to wait to introduce if</code>-let until much later like we do now</a>.</p> Closing Words</h2> Another thing I wasn't quite sure when to talk about when it comes to the guard patterns RFC is its interaction with pattern types. Bringing if</code>-guards into pattern feels like it changes the pattern notation from a fairly tractable/limited subset that we can evaluate in reasonable time into a system for arbitrary pre/post-conditions. Unless of course we disallow those kinds of patterns in pattern types - which creates diverging pattern notations.</p> Overall I feel like we have a lot to gain in the language by making match</code>-expressions feel more consistent with the rest of the conditional expressions. After writing the bulk of this post, I remembered that what I'm sharing here is actually not a unique insight. RFC 3796</a> cfg_logical_ops</code> by Jacob Pratt proposes the addition of the boolean &&</code>, \|\|</code>, and !</code> operators in cfg</code> attributes. Effectively replacing the existing domain-specific all()</code>, any()</code>, and not()</code> operators.</p> // current </span>#[</span>cfg</span>(</span>all</span>(</span>any</span>(a, b), </span>any</span>(c, d), </span>not</span>(e)))] </span>struct </span>MyStruct {} </span> </span>// proposed </span>#[</span>cfg</span>((a \|\| b) && (</span>c</span> \|\| </span>d</span>) && !</span>e</span>)] </span>struct </span>MyStruct {} </span></code></pre> I like what that RFC proposes. I doubly like it when we consider it as part of a broader effort to reduce the amount of micro-syntaxes</a> spread throughout the language. In turn making Rust an easier language, by virtue of it being smaller and more consistent.</p> Syntactic Musings on View Types Fri, 04 Apr 2025 00:00:00 +0000 Here's a silly little insight I had the other day: if you squint, both View Types and Pattern Types seem like lightweight forms of Refinement Types 1</a></sup>. Both will enable constraining types, but in slightly different and complementary ways. Let's take a look at this using an example of an RGB struct containing fields for individual Red, Green, and Blue channels stored as usize</code> 2</a></sup>:</p> ^{1</sup> I like the term "lightweight refinement types" for the category of extensions that include pattern types and view types. To me it's reminiscent of lightweight formal methods</em>: less thorough than the full thing, but with incredible returns for the relative effort expended.</p> </div>}^{2</sup> Don't question why we're storing these values as usize</code> too much. It's a little silly. For the purpose of this example just pretend there is some reason why we have to do it like this.</p> </div>}#[</span>derive</span>(Debug, Default)] </span>struct </span>Rgb { </span> </span>r</span>: </span>usize</span>, </span> </span>g</span>: </span>usize</span>, </span> </span>b</span>: </span>usize</span>, </span>} </span></code></pre> Pattern types will give us the ability to directly use things like ranges or enum members in type signatures. In the following example the type usize</code> is refined</em> to statically only be allowed to contain values between 0..255</code>. This is similar to the NonZero</code> types in the stdlib, but as part of the language and usable with arbitrary patterns:</p> impl </span>Rgb { </span> </span>fn </span>set_red</span>(&</span>mut </span>self</span>, </span>num</span>: </span>usize</span> is ..255) { .. } </span> </span>// ^^^^^^^^ range pattern </span>} </span></code></pre> View types are about segmenting the fields contained in self</code> so that multiple (mutable) methods can operate on the same type without resulting in borrow checking issues. In the following example we provide mutable access to individual fields using separate methods. None of these methods take overlapping borrows of the fields in Self</code>. This means we're free to call all of these methods, observe their return types, and we won't get any borrow checker errors. Here's an example using the syntax from Niko's latest post</a>:</p> impl </span>Rgb { </span> </span>fn </span>red_mut</span>(</span>self</span>: &</span>mut</span> { r } </span>Self</span>) -> .. { .. } </span> </span>// ^^^^^ view </span> </span>fn </span>green_mut</span>(</span>self</span>: &</span>mut</span> { g } </span>Self</span>) -> .. { .. } </span> </span>// ^^^^^ view </span> </span>fn </span>blue_mut</span>(</span>self</span>: &</span>mut</span> { b } </span>Self</span>) -> .. { .. } </span> </span>// ^^^^^ view </span>} </span></code></pre> Here's a fun question: what happens if we combine combine Pattern Types and View Types? Both serve different purposes, and I know I've got cases where I'd like to combine them. So what would that look like? In the abstract, it seems like we would end up with something like the following:</p> impl </span>Rgb { </span> </span>fn </span>foo</span>(</span>self</span>: &</span>mut</span> { r, g } </span>Self</span> is </span>Self</span> { </span>r</span>: ..255, </span>g</span>: ..255, .. }) {} </span> </span>// ^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ </span> </span>// view pattern </span>} </span></code></pre> To me that seems like a lot to read. But also rather... double? Both View Types and Pattern Types refine Self</code> here. I would have expected us to be able to combine both. Now what if View Types and Pattern Types instead shared the same syntactic position. There's a reason View Types have to use is</code>, so let's use that. With that we could rewrite our earlier example like this instead:</p> impl </span>Rgb { </span> </span>fn </span>foo</span>(&</span>mut </span>self</span> is </span>Self</span> { </span>r</span>: ..255, </span>g</span>: ..255, .. }) {} </span> </span>// ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ </span> </span>// view + pattern </span>} </span></code></pre> This seems a lot more readable to me. Combining both features seems much less like a hassle here, and maybe even… nice? This updated notation would also affect our earlier View Types example. Using the updated notation would now be written the same way, but without using any patterns:</p> impl </span>Rgb { </span> </span>fn </span>red_mut</span>(&</span>mut </span>self</span> is </span>Self</span> { r, .. }) -> .. { .. } </span> </span>// ^^^^^^^^^ view </span> </span>fn </span>green_mut</span>(&</span>mut </span>self</span> is </span>Self</span> { g, .. }) -> .. { .. } </span> </span>// ^^^^^^^^^ view </span> </span>fn </span>blue_mut</span>(&</span>mut </span>self</span> is </span>Self</span> { b, .. }) -> .. { .. } </span> </span>// ^^^^^^^^^ view </span>} </span></code></pre> I feel like this would rather neatly unify where we talk about refinements</em> in function signatures. Notationally this reframes View Types as Pattern Types with ignored fields. Though we don't necessarily need to adopt this notation: what I care about most is that we think about how both features are intended to come together in the language to create a cohesive experience. Thanks for reading!</p> A survey of every iterator variant Wed, 12 Feb 2025 00:00:00 +0000 Introduction</h2> Oh yeah, you like iterators? Name all of them. —</p> </blockquote> I'm increasingly of the belief that before we can meaningfully discuss the solution space, we have to first make an effort to build consensus on the problem space. It's hard to plan for a journey together if we don't know what the trip will be like along the way. Or worse: if we don't agree in advance where we want the journey to take us.</p> In Rust the Iterator</code> trait is the single-most complex trait in the stdlib. It provides 76 methods, and by my estimate (I stopped counting at 120) has around 150 trait implementations in the stdlib alone. It also features a broad range of extension traits like FusedIterator</code> and ExactSizeIterator</code> that provide additional capabilities. And it is itself a trait-based expression of one of Rust's core control-flow effects; meaning it is at the heart of a lot of interesting questions about how we combine such effects.</p> Despite all that we know the Iterator</code> trait falls short today and we would like to extend it to do more</em>. Async iteration is one popular capability people are missing as a built-in. Another is the ability to author address-sensitive (self-referential) iterators. Less in the zeitgeist but equally important are iterators that can lend items with a lifetime, conditionally terminate early, and take external arguments on each call to next</code>.</p> I'm writing this post to enumerate all of the iterator variants in use in Rust today. So that we can scope the problem space to account for all of them. I'm also doing this because I've started hearing talk about potentially (soft-)deprecating Iterator</code>. I believe we only get one shot at running such a deprecation 1</a></sup>, and if we are going to do one we need to make sure it's a solution that will plausibly get us through all of our known limitations. Not just one or two.</p> So without further ado: let's enumerate every variation of the existing Iterator</code> trait we know we want to write, and discuss how those variants interact with each other to create interesting new kinds of iterators.</p> ^{1</sup> I'm basing this off of my experience being a part of the Node.js Streams WG through the mid-2010s. By my count Node.js now features five distinct generations of stream abstractions. Unsurprisingly it's not particularly pleasant to use, and ideally something we should avoid replicating in Rust. Though I'm not opposed to running deprecations of core traits in Rust either, I believe that in order to do that we want to be approaching 100% certainty that it'll be the last deprecation we do. If we're going to embark on a decade plus of migration issues, we have to make absolutely sure it's worth it.</p> </div>}Base Iterator</h2> Iterator</code> is a trait representing the stateful component of iteration; IntoIterator</code> represents the capability for a type to be iterated over. Here is the Iterator</code> trait as found in the stdlib today. The Iterator</code> and IntoIterator</code> traits are closely linked, so throughout this post we will show both to paint a more complete picture.</p> Throughout this post we will be covering variations on Iterator</code> which provide new capabilities. This trait is represents the absence of those capabilities: it is blocking, cannot be evaluated during compilation, is strictly sequential, and so on. Here is what the foundation of the core::iter</code> submodule looks like today:</p> /// An iterable type. </span>pub trait </span>IntoIterator { </span> </span>/// The type of elements yielded from the iterator. </span> </span>type </span>Item; </span> </span>/// Which kind of iterator are we turning this into? </span> </span>type </span>IntoIter: Iterator<Item = </span>Self::</span>Item>; </span> </span>/// Returns an iterator over the elements in this type. </span> </span>fn </span>into_iter</span>(</span>self</span>) -> </span>Self::</span>IntoIter; </span>} </span> </span>/// A type that yields values, one at the time. </span>pub trait </span>Iterator { </span> </span>/// The type of elements yielded from the iterator. </span> </span>type </span>Item; </span> </span>/// Advances the iterator and yields the next value. </span> </span>fn </span>next</span>(&</span>mut </span>self</span>) -> Option<</span>Self::</span>Item>; </span> </span>/// Returns the bounds on the remaining length of the iterator. </span> </span>fn </span>size_hint</span>(&</span>self</span>) -> (</span>usize</span>, Option<</span>usize</span>>) { .. } </span>} </span></code></pre> While not strictly necessary: the associated IntoIterator::Item</code> type exists for convenience. That way people using the trait can directly specify the Item</code> using impl IntoIterator<Item = Foo></code>, which is a lot more convenient than I: <IntoIterator::IntoIterator as Iterator>::Item</code>, etc.</p> Bounded Iterator</h2> The base Iterator</code> trait represents a potentially infinite (unbounded) sequence of items. It has a size_hint</code> method that returns how many items the iterator still expects to yield. That value is however not guaranteed to be correct, and is intended to be used for optimizations only. From the docs:</p> Implementation notes</strong> It is not enforced that an iterator implementation yields the declared number of elements. A buggy iterator may yield less than the lower bound or more than the upper bound of elements.</p> size_hint()</code> is primarily intended to be used for optimizations such as reserving space for the elements of the iterator, but must not be trusted to e.g., omit bounds checks in unsafe code. An incorrect implementation of size_hint()</code> should not lead to memory safety violations.</p> </blockquote> The Rust stdlib provides two Iterator</code> subtraits that allow it to guarantee it is bounded: ExactSizeIterator</code> (stable) and TrustedLen</code> (unstable). Both traits take : Iterator</code> as its super-trait bound and on its surface seem nearly identical. But there is one key difference: TrustedLen</code> is unsafe to implement allowing it to be used to guarantee safety invariants.</p> /// An iterator that knows its exact length. </span>pub trait </span>ExactSizeIterator: Iterator { </span> </span>/// Returns the exact remaining length of the iterator. </span> </span>fn </span>len</span>(&</span>self</span>) -> </span>usize </span>{ .. } </span> </span>/// Returns `true` if the iterator is empty. </span> </span>fn </span>is_empty</span>(&</span>self</span>) -> </span>bool </span>{ .. } </span>} </span> </span>/// An iterator that reports an accurate length using `size_hint`. </span>#[</span>unstable</span>(feature = "</span>trusted_len</span>")] </span>pub unsafe trait </span>TrustedLen: Iterator {} </span></code></pre> ExactSizeIterator</code> has the same memory-safety guarantees as Iterator::size_hint</code> meaning: you can't rely on it to be correct. That means that if you're say, collecting items from an iterator into a vec, you can't omit bounds checks and use ExactSizeIterator::len</code> as the input to Vec::set_len</code></a>. However if TrustedLen</code> is implemented bounds checks can be omitted, since the value returned by size_hint</code> is now a safety invariant of the iterator.</p> Fused Iterator</h2> When working with an iterator, we typically iterate over items until the iterator yields None</code> at which point we treat the iterator as "done". However the documentation for Iterator::next</code> includes the following:</p> Returns None</code> when iteration is finished. Individual iterator implementations may choose to resume iteration, and so calling next()</code> again may or may not eventually start returning Some(Item)</code> again at some point.</p> </blockquote> It's rare to work with iterators which yield None</code> and then resume again afterwards, but the iterator trait explicitly allows it. Just like it allows for an iterator to panic if next</code> is called again after None</code> has been yielded once. Luckily the FusedIterator</code> subtrait exists, which guarantees that once None</code> has been yielded once from an iterator, all future calls to next</code> will continue to yield None</code>.</p> /// An iterator that always continues to yield `None` when exhausted. </span>pub trait </span>FusedIterator: Iterator {} </span></code></pre> Most iterators in the stdlib implement FusedIterator</code>. For iterators which aren't fused, it's possible to call the Iterator::fuse</code> combinator. The stdlib docs recommend never taking a FusedIterator</code> bound, instead favoring Iterator</code> in bounds and calling fuse</code> to guarantee the fused behavior. For iterators that already implement FusedIterator</code> this is considered a no-op.</p> The FusedIterator</code> design works because Option::None</code> is idempotent</em>: there is never a scenario where we can't create a new instance of None</code>. Contrast this with enums such as Poll</code></a> that lack a "done" state - and you see ecosystem traits such as FusedFuture</code></a> attempting to add this lack of expressivity back through other means. The need for an idempotent "done" state will be important to keep in mind as we explore other iterator variants throughout this post.</p> Thread-Safe Iterator</h2> Where bounded and fused iterators can be obtained by refining the Iterator</code> trait using subtraits, thread-safe iterators are obtained by composing the Send</code> and Sync</code> auto-traits with the Iterator</code> trait. That means there is no need for dedicated SendIterator</code> or SyncIterator</code> traits. A "thread-safe iterator" becomes a composition of Iterator</code> and Send</code> / Sync</code>:</p> struct </span>Cat; </span> </span>// Manual `Iterator` impl </span>impl </span>Iterator </span>for </span>Cat { .. } </span> </span>// These impls are implied by `Send` </span>// and `Sync` being auto-traits </span>unsafe impl </span>Send </span>for </span>Cat {} </span>unsafe impl </span>Sync </span>for </span>Cat {} </span></code></pre> And when taking impls in bounds, we can again express our intent by composing traits in bounds. I've made the argument before that taking Iterator</code> directly in bounds is rarely what people actually want, so the bound would end up looking like this:</p> fn </span>thread_safe_sink</span>(</span>iter</span>: impl IntoIterator + Send) { .. } </span></code></pre> If we also want the individual items yielded by the iterator to be marked thread-safe, we have to add additional bounds:</p> fn </span>thread_safe_sink</span><I>(</span>iter</span>: I) </span>where </span> I: IntoIterator<Item: Send> + Send, </span>{ .. } </span></code></pre> While there is no lack of expressivity here, at times bounds like these can get rather verbose. That should not be taken as indictment of the system itself, but rather as a challenge to improve the ergonomics for the most common cases.</p> Dyn-Compatible Iterator</h2> Dyn-compatibility is another axis that traits are divided on. Unlike e.g. thread-safety, dyn-compatibility is an inherent part of the trait and is governed by Sized</code> bounds. Luckily both the Iterator</code> and IntoIterator</code> traits are inherently dyn-compatible. That means they can be used to create trait objects using the dyn</code> keyword:</p> struct </span>Cat; </span>impl </span>Iterator </span>for </span>Cat { .. } </span> </span>let</span> cat = Cat {}; </span>let</span> dyn_cat: &dyn Iterator = &cat; </span>// ok </span></code></pre> There are some iterator combinators such as count</code></a> that take an additional Self: Sized</code> bound 2</a></sup>. But because trait objects are themselves sized, it all mostly works as expected:</p> ^{2</sup> Admittedly my knowledge of dyn</code> is spotty at best. Out of everything in this post, the section on dyn</code> was the bit I had the least intuition about.</p> </div>}let mut</span> cat = Cat {}; </span>let</span> dyn_cat: &</span>mut</span> dyn Iterator = &</span>mut</span> cat; </span>assert_eq!(dyn_cat.</span>count</span>(), </span>1</span>); </span>// ok </span></code></pre> Double-Ended Iterator</h2> Often times iterators over collections hold all data in memory, and can be traversed in either direction. For this purpose Rust provides the DoubleEndedIterator</code> trait. Where Iterator</code> builds on the next</code> method, DoubleEndedIterator</code> builds on a next_back</code> method. This allows items to be taken from both the logical start and end of the iterator. And once both cursors meet, iteration is considered over.</p> /// An iterator able to yield elements from both ends. </span>pub trait </span>DoubleEndedIterator: Iterator { </span> </span>/// Removes and an element from the end of the iterator. </span> </span>fn </span>next_back</span>(&</span>mut </span>self</span>) -> Option<</span>Self::</span>Item>; </span>} </span></code></pre> While you'd expect this trait to be implemented for e.g. VecDeque</code>, it's interesting to note that it's also implemented for Vec</code>, String</code>, and other collections that only grow in one direction. Also unlike some of the other iterator refinements we've seen, DoubleEndedIterator</code> has a required method that is used as the basis for several new methods such as rfold</code></a> (reverse fold</code>) and rfind</code></a> (reverse find</code>).</p> Seeking Iterator</h2> Both the Iterator</code> and Read</code>traits in Rust provide abstractions for streaming iteration. The main difference is that Iterator</code> works by returning arbitrary owned types when calling next</code>, where Read</code> is limited to reading bytes into buffers. But both abstractions keep a cursor that keeps track of which data has been processed already, and which data still needs processing.</p> But not all streams are created equally. When we read data from a regular file 3</a></sup> on disk we can be sure that, as long as no writes occurred, we can read the same file again and get the same output. That same guarantee is however not true for sockets, where once we've read data from it we can't read that same data again. In Rust this distinction is surfaced via the Seek</code> trait, which gives control over the Read</code> cursor in types that support it.</p> ^{3</sup> No, files in /proc</code> are not "regular files". Yes, I know sockets are technically file descriptors.</p> </div>}In Rust the Iterator</code> trait provides no mechanism to control the underlying cursor, despite its similarities to Read</code>. A language that does provide an abstraction for this is C++ in the form of random_access_iterator</code></a>. This is a C++ concept (trait-alike) that further refines bidirectional_iterator</code>. My C++ knowledge is limited, so I'd rather quote the docs directly than attempt to paraphrase:</p> [...] random_access_iterator refines bidirectional_iterator by adding support for constant time advancement with the +=</code>, +</code>, -=</code>, and -</code> operators, constant time computation of distance with -</code>, and array notation with subscripting []</code>.</p> </blockquote> Being able to directly control the cursor in Iterator</code> implementations could prove useful when working with in-memory collection types like Vec</code>, as well as when working with remote objects such as paginated API endpoints. The obvious starting point for such a trait would be to mirror the existing io::Seek</code> trait and adapt it to be a subtrait of Iterator</code>:</p> /// Enumeration of possible methods to seek within an iterator. </span>pub enum </span>SeekFrom { </span> </span>/// Sets the offset to the specified index. </span> Start(</span>usize</span>), </span> </span>/// Sets the offset to the size of this object plus the specified index. </span> End(</span>isize</span>), </span> </span>/// Sets the offset to the current position plus the specified index. </span> Current(</span>isize</span>), </span>} </span> </span>/// An iterator with a cursor which can be moved. </span>pub trait </span>SeekingIterator: Iterator { </span> </span>/// Seek to an offset in an iterator. </span> </span>fn </span>seek</span>(&</span>mut </span>self</span>, </span>pos</span>: SeekFrom) -> Result<</span>usize</span>>; </span>} </span></code></pre> Compile-Time Iterator</h2> In Rust we can use const {}</code> blocks to execute code during compilation. Only const fn</code> functions can be called from const {}</code> blocks. const fn</code> free functions and methods are stable, but const fn</code> trait methods are not. This means that traits like Iterator</code> are not yet callable from const {}</code> blocks, and so neither are for..in</code> expressions.</p> We know we want to support iteration in const {}</code> blocks, but we don't yet know how we want to spell both the trait declaration and trait bounds. The most interesting open question here is how we will end up passing the Destruct</code> trait</a> around, which is necessary to enable types to be droppable in const</code> contexts. This leads to additional questions around whether const trait bounds should imply const Destruct</code>. And whether the const</code> annotation should be part of the trait declaration, the individual methods, or perhaps both.</p> This post is not the right venue to discuss all tradeoffs. But to give a sense of what a compile-time-compatible Iterator</code> trait might look like: here is a variant where both the trait and individual methods are annotated with const</code>:</p> pub const trait </span>IntoIterator { </span>// ← `const` </span> </span>type </span>Item; </span> </span>type </span>IntoIter: </span>const </span>Iterator<Item = </span>Self::</span>Item>; </span>// ← `const` </span> </span>const fn </span>into_iter</span>(</span>self</span>) -> </span>Self::</span>IntoIter; </span>// ← `const` </span>} </span> </span>pub const trait </span>Iterator { </span>// ← `const` </span> </span>type </span>Item; </span> </span>const fn </span>next</span>(&</span>mut </span>self</span>) -> Option<</span>Self::</span>Item>; </span>// ← `const` </span> </span>const fn </span>size_hint</span>(&</span>self</span>) -> (</span>usize</span>, Option<</span>usize</span>>) { .. } </span>// ← `const` </span>} </span></code></pre> Lending Iterator</h2> While there are plenty of iterators in the stdlib today that yield references to items, the values that are being referenced are never owned by the iterator itself. In order to write iterators which owns the items it yields and yields them by-reference, the associated item type in iterator needs a lifetime:</p> pub trait </span>IntoIterator { </span> </span>type </span>Item<</span>'a</span>> </span>// ← Lifetime </span> </span>where </span> </span>Self</span>: </span>'a</span>; </span>// ← Bound </span> </span>type </span>IntoIter: </span>for</span><</span>'a</span>> Iterator<Item<</span>'a</span>> = </span>Self::</span>Item<</span>'a</span>>>; </span>// ← Lifetime </span> </span>fn </span>into_iter</span>(</span>self</span>) -> </span>Self::</span>IntoIter; </span>} </span> </span>pub trait </span>Iterator { </span> </span>type </span>Item<</span>'a</span>> </span>// ← Lifetime </span> </span>where </span> </span>Self</span>: </span>'a</span>; </span>// ← Bound </span> </span>fn </span>next</span>(&</span>mut </span>self</span>) -> Option<</span>Self::</span>Item<'_>>; </span>// ← Lifetime </span> </span>fn </span>size_hint</span>(&</span>self</span>) -> (</span>usize</span>, Option<</span>usize</span>>) { .. } </span>} </span></code></pre> There has been talk about adding a 'self</code> lifetime to the language as a shorthand for where Self:'a</code>. But even with that addition this set of signatures is not for the faint of heart. The reason for that is that it makes use of GATs, which are both useful and powerful — but are for all intents and purposes an expert-level type-system feature, and can be a little tricky to reason about.</p> Lending iteration is also going to be important when we add dedicated syntax to construct iterators using the yield</code> keyword. The following example shows a gen {}</code> block which creates a string and then yields a reference to it. This perfectly4</a></sup> matches onto the Iterator</code> trait we’ve defined:</p> ^{4</sup> Remember that strings are heap-allocated so we don’t need to account for address-sensitivity. While the compiler can't yet perform this desugaring, there is nothing in the language prohibiting it.</p> </div>}let</span> iter = gen { </span> </span>let</span> name = String::from("</span>Chashu</span>"); </span> </span>yield</span> &name; </span>// ← Borrows a local value stored on the heap. </span>}; </span></code></pre> Iterator with a Return Value</h2> The current Iterator</code> trait has an associated type Item</code>, which maps to the yield</code> keyword in Rust. But it has no associated type that maps to the return</code> keyword. A way to think about iterators today is that their return type is hard-coded to unit. If we want to enable generator functions and blocks to be written which can not just yield</code>, but also return</code>, we'll need some way to express that.</p> let</span> counting_iter = gen { </span> </span>let mut</span> total = </span>0</span>; </span> </span>for</span> item in iter { </span> total += </span>1</span>; </span> </span>yield</span> item; </span>// ← Yields one type. </span> } </span> total </span>// ← Returns another type. </span>}; </span></code></pre> The obvious way to write a trait for "an iterator which can return" is to give Iterator</code> an additional associated item Output</code> which maps to the logical return value. In order to be able to express fuse semantics, the function next</code> needs to be able to return three different states:</p> Yielding the associated type Item</code></li> Returning the associated type Output</code></li> The iterator has been exhausted (done)</li> </ol> One way to do this would be for next</code> to return Option<ControlFlow></code>, where Some(Continue)</code> maps to yield</code>, Some(Break)</code> maps to return</code>, and None</code> maps to done. Without a final "done" state, calling next</code> again on an iterator after it's finished yielding values would likely always have to panic. This is what most futures do in async Rust, and is going to be a problem if we ever want to guarantee panic-freedom.</p> pub trait </span>IntoIterator { </span> </span>type </span>Output; </span>// ← Output </span> </span>type </span>Item; </span> </span>type </span>IntoIter: </span> Iterator<Output = </span>Self::</span>Output, Item = </span>Self::</span>Item>; </span>// ← Output </span> </span>fn </span>into_iter</span>(</span>self</span>) -> </span>Self::</span>IntoIter; </span>} </span> </span>pub trait </span>Iterator { </span> </span>type </span>Output; </span>// ← Output </span> </span>type </span>Item; </span> </span>fn </span>next</span>(&</span>mut </span>self</span>) </span> -> Option<ControlFlow<</span>Self::</span>Output, </span>Self::</span>Item>>; </span>// ← Output </span> </span>fn </span>size_hint</span>(&</span>self</span>) -> (</span>usize</span>, Option<</span>usize</span>>) { .. } </span>} </span></code></pre> The way this would work on the caller side and in combinators is quite interesting. Starting with the provided for_each</code></a> method: it will want to return Iterator::Output</code> rather than ()</code> once the iterator has completed. Crucially the closure F</code> provided to for_each</code> only operates on Self::Output</code>, and has no knowledge of Self::Output</code>. Because if the closure did have direct knowledge of Output</code>, it could short-circuit by returning Output</code> sooner than expected which is a different kind of iterator than having a return value.</p> fn </span>for_each</span><F>(</span>self</span>, </span>f</span>: F) -> </span>Self::</span>Output </span>// ← Output </span>where </span> </span>Self</span>: Sized, </span> F: FnMut(</span>Self::</span>Item), </span>{ .. } </span></code></pre> If we were to transpose "iterator with return value" to for..in</code> things get even more interesting still. In Rust loop</code> expressions can themselves evaluate to non-unit types by calling break</code> with some value. for..in</code> expressions cannot do this in the general case yet, except for the handling of errors using ?</code>. But it's not hard to see how this could be made to work, conceptually this is equivalent to calling Iterator::try_for_each</code></a> and returning Some(Break(value))</code>:</p> let</span> ty: </span>u32 </span>= </span>for</span> item in iter { </span>// ← Evaluates to a `u32` </span> </span>break </span>12</span>u32 </span>// ← Calls `break` from the loop body </span>}; </span></code></pre> Assuming we have an Iterator</code> with its own return value, that would mean for..in</code> expressions would be able to evaluate to non-unit return types without calling break</code> from within the loop's body:</p> let</span> ty: </span>u32 </span>= </span>for</span> item in iter { </span>// ← Evaluates to a `u32` </span> dbg!(item); </span>// ← Doesn't `break` from the loop body </span>}; </span></code></pre> This of course leads to questions about how to combine an "iterator with a return value" and "using break</code> from inside a for..in</code> expression". I'll leave that as an exercise to the reader on how to spell out (I'm certain it can be done, I just think it's a fun one). Generalizing all modes of early returns from for..in</code> expressions to invocations of for_each</code> combinators is an interesting challenge that we'll cover in more detail later on when we discuss short-circuiting (fallible) iterators.</p> Iterator with a Next Argument</h2> In generator blocks the yield</code> keyword can be used to repeatedly yield values from an iterator. But what if the caller doesn't just want to obtain values, but also pass new values back into the iterator? That would require for yield</code> to be able evaluate to a non-unit type. Iterators that have this functionality are often referred to as "coroutines", and they are being particularly useful when implementing I/O-Free Protocols</a>.</p> /// Some RPC protocol </span>enum </span>Proto { </span> </span>/// Some protocol state </span> MsgLen(</span>u32</span>), </span>} </span> </span>let</span> rpc_handler = gen { </span> </span>let</span> len = message.</span>len</span>(); </span> </span>let</span> next_state = </span>yield</span> Proto::MsgLen(len); </span>// ← `yield` evaluates to a value </span> .. </span>}; </span></code></pre> In order to support this, Iterator::next</code> needs to be able to take an additional argument in the form of a new associated type Args</code>. This associated type has the same name as the input arguments to Fn</code> and AsyncFn</code>. If "iterator with a next argument" can be thought of as representing a "coroutine", the Fn</code> family of traits can be thought of as representing a regular "routine" (function).</p> pub trait </span>IntoIterator { </span> </span>type </span>Item; </span> </span>type </span>Args; </span>// ← Args </span> </span>type </span>IntoIter: </span> Iterator<Item = </span>Self::</span>Item, Args = </span>Self::</span>Args>; </span>// ← Args </span> </span>fn </span>into_iter</span>(</span>self</span>) -> </span>Self::</span>IntoIter; </span>} </span> </span>pub trait </span>Iterator { </span> </span>type </span>Item; </span> </span>type </span>Args; </span>// ← Args </span> </span>fn </span>next</span>(&</span>mut </span>self</span>, </span>args</span>: </span>Self::</span>Args) -> Option<</span>Self::</span>Item>; </span>// ← Args </span> </span>fn </span>size_hint</span>(&</span>self</span>) -> (</span>usize</span>, Option<</span>usize</span>>) { .. } </span>} </span></code></pre> Here too it's interesting to consider the caller side. Starting with a hypothetical for_each</code> function: it would need to be able to take an initial value passed to next</code>, and then from the closure be able to return additional values passed to subsequent invocations of next</code>. The signature for that would look like this:</p> fn </span>for_each</span><F>(</span>self</span>, </span>args</span>: </span>Self::</span>Args, </span>f</span>: F) </span>// ← Args </span>where </span> </span>Self</span>: Sized, </span> F: FnMut(</span>Self::</span>Item) -> </span>Self::</span>Args, </span>// ← Args </span>{ .. } </span></code></pre> To better illustrate how this would work and build up some intuition, consider manual calls to next</code> inside of a loop. We would start by constructing some initial state that is passed to the first invocation of next</code>. This will produce an item that can be used to construct the next argument to next</code>. And so on, until the iterator has no more items left to yield:</p> let mut</span> iter = some_value.</span>into_iter</span>(); </span>let mut</span> arg = some_initial_value; </span> </span>// loop as long as there are items to yield </span>while let </span>Some(item) = iter.</span>next</span>(arg) { </span> </span>// use `item` and compute the next `arg` </span> arg = </span>process_item</span>(item); </span>} </span></code></pre> If we were to iterate over an iterator with a next function using a for..in</code> expression, the equivalent of returning a value from a closure would be either to continue</code> with a value. This could potentially also be the final expression in the loop body, which you can think of itself being an implied continue</code> today. The only question remaining is how to pass initial values on loop construction, but that mainly seems like an exercise in syntax design:</p> // Passing a value to the `next` function seems like </span>// it would logically map to `continue`-expressions. </span>// (passing initial state to the loop intentionally omitted) </span>for</span> item in iter { </span> </span>continue </span>process_item</span>(item); </span>// ← `continue` with value </span>}; </span> </span>// You can think of `for..in` expressions as having </span>// an implied `continue ()` at the end. Like functions </span>// have an implied `return ()`. What if that could take </span>// a value? </span>// (passing initial state to the loop intentionally omitted) </span>for</span> item in iter { </span> </span>process_item</span>(item) </span>// ← `continue` with value </span>}; </span></code></pre> "iterator with return value" and "iterator with next argument" feel like they map particularly well to break</code> and continue</code>. I think of them as duals, enabling both expressions to carry non-unit types. This feels like it might be an important insight that I haven't seen anyone bring up before.</p> Being able to pass a value to next</code> is one of the hallmark features of the Coroutine</code> trait in the stdlib. However unlike the trait sketch we provided here, in Coroutine</code> the type of the value passed to next</code> is defined as a generic param on the trait rather than an associated type. Presumably that is so that Coroutine</code> can have multiple implementations on the same type that depend on the input type. I searched whether this was actually being used today, and it doesn't seem to be</a>. Which is why I suspect it is likely fine to use an associated type for the input arguments.</p> Short-Circuiting Iterator</h2> While it is possible to return Try</code>-types such as Result</code> and Option</code> from an iterator today, it isn't yet possible is to immediately stop execution in the case of an error [^not-panic]. This behavior is typically referred to as "short-circuiting": halting normal operation and triggering an exceptional state (like an electrical breaker would in a building).</p> ^{5</sup> By that I mean: an error, not a panic.</p> </div> In unstable Rust we have try {}</code> blocks that rely on the unstable Try</code> trait, but we don't yet have try fn</code> functions. If we want these to function in traits the way we'd want them to, they will have to desugar to impl Try</code>. Rather than speculate about what a potential try fn</code> syntax might look like in the future, we'll be writing our samples using -> impl Try</code> directly. Here is what the Try</code> (and FromResidual</code>) traits look like today:</p>}/// The `?` operator and `try {}` blocks. </span>pub trait </span>Try: FromResidual { </span> </span>/// The type of the value produced by `?` </span> </span>/// when _not_ short-circuiting. </span> </span>type </span>Output; </span> </span> </span>/// The type of the value passed to `FromResidual::from_residual` </span> </span>/// as part of ? when short-circuiting. </span> </span>type </span>Residual; </span> </span> </span>/// Constructs the type from its `Output` type. </span> </span>fn </span>from_output</span>(</span>output</span>: </span>Self::</span>Output) -> </span>Self</span>; </span> </span> </span>/// Used in ? to decide whether the operator should produce </span> </span>/// a value ( or propagate a value back to the caller. </span> </span>fn </span>branch</span>(</span>self</span>) -> ControlFlow<</span>Self::</span>Residual, </span>Self::</span>Output>; </span>} </span> </span>/// Used to specify which residuals can be converted </span>/// into which `Try` types. </span>pub trait </span>FromResidual<R = <Self as Try>::Residual> { </span> </span>/// Constructs the type from a compatible `Residual` type. </span> </span>fn </span>from_residual</span>(</span>residual</span>: R) -> </span>Self</span>; </span>} </span></code></pre> You can think of a short-circuiting iterator as a special case of an "iterator with return value". In its base form, it will only return early in case of an exception while its logical return type remains hard-coded to unit. The return type of fn next</code> should be an impl Try</code> returning an Option</code>, with the value of Residual</code> set to the associated Residual</code> type. This allows all combinators to share the same Residual</code>, enabling types to flow.</p> pub trait </span>IntoIterator { </span> </span>type </span>Item; </span> </span>type </span>Residual; </span> </span>type </span>IntoIter: Iterator< </span> Item = </span>Self::</span>Item, </span> Residual = Residual </span>// ← Residual </span> >; </span> </span>fn </span>into_iter</span>(</span>self</span>) -> </span>Self::</span>IntoIter; </span>} </span> </span>pub trait </span>Iterator { </span> </span>type </span>Item; </span> </span>type </span>Residual; </span>// ← Residual </span> </span>fn </span>next</span>(&</span>mut </span>self</span>) -> impl Try< </span>// ← impl Try </span> Output = Option<</span>Self::</span>Item>, </span> Residual = </span>Self::</span>Residual, </span>// ← Residual </span> >; </span>} </span></code></pre> If we once again consider the caller, we'll want to provide a way for for..in</code> expressions to short-circuit. What's interesting here is that the base iterator trait already provides a try_for_each</code></a>, method. The difference between that method and the for_each</code> we're about to see is how the type of Residual</code> is obtained. In try_for_each</code> the value is local to the method, while if the trait itself is "short-circuiting" the type of Residual</code> is determined by the associated Self::Residual</code> type. Or put differently: in a short-circuiting iterator the type we short-circuit with is a property of the trait rather than a property of the method.</p> fn </span>for_each</span><F, R>(</span>self</span>, </span>f</span>: F) -> R </span>// ← Return type </span>where </span> </span>Self</span>: Sized, </span> F: FnMut(</span>Self::</span>Item) -> R, </span>// ← Return type </span> R: Try<Output = (), Residual = </span>Self::</span>Residual>, </span>// ← `impl Try` </span>{ .. } </span></code></pre> As mentioned earlier on in this post: the interaction between "iterator with a return type" and "short-circuiting iterator" is an interesting one. Returning Option<ControlFlow></code> from fn next</code> is able to encode three distinct states, but this combination of capabilities requires us to encode four states:</p> yield a next item</li> break with a residual</li> return a final output</li> iterator done (idempotent)</li> </ol> The reason why we want to be able to encode a signature like this is because when writing gen fn</code> functions it is entirely reasonable to want to have a return type, short-circuit on error using ?</code>, and also yield</code> values. This works like regular functions today, but with the added capability to invoke yield</code>. The naive way of encoding this would be to return an impl Try</code> of Option<ControlFlow<_>></code> with distinct associated types for Item</code>, Output</code>, and Residual</code>. This does however feel like it is starting to get a little out of hand, though perhaps a first-class try fn</code> notation might provide some relief.</p> pub trait </span>IntoIterator { </span> </span>type </span>Item; </span> </span>type </span>Output; </span>// ← `Output` </span> </span>type </span>Residual; </span>// ← `Residual` </span> </span>type </span>IntoIterator: Iter< </span> Item = </span>Self::</span>Item, </span> Residual = </span>Self::</span>Residual, </span>// ← `Residual` </span> Output = </span>Self::</span>Output, </span>// ← `Output` </span> >; </span> </span>fn </span>into_iter</span>(</span>self</span>) -> </span>Self::</span>IntoIter; </span>} </span> </span>pub trait </span>Iterator { </span> </span>type </span>Item; </span> </span>type </span>Output; </span>// ← `Output` </span> </span>type </span>Residual; </span>// ← `Residual` </span> </span>fn </span>next</span>(&</span>mut </span>self</span>) -> impl Try< </span>// ← `impl Try` </span> Output = Option<ControlFlow< </span>// ← `ControlFlow </span> </span>Self::</span>Output, </span>// ← `Output </span> </span>Self::</span>Item, </span> >>, </span> Residual = </span>Self::</span>Residual, </span>// ← `Residual` </span> >; </span>} </span></code></pre> Address-Sensitive Iterator</h2> Rust's generator transformation may create self-referential types. That is: types which have fields that borrow from other fields on the same type. We call these types "address-sensitive" because once the type has been constructed, its address in memory must remain stable. This comes up when writing gen {}</code> blocks that have stack-allocated locals 6</a></sup> that are kept live across yield</code>-expressions. What is or isn't a "stack-allocated local" can get a little complicated. But it's important to highlight that for example calling IntoIterator::into_iter</code> on a type and re-yielding all items is something that just works (playground</a>):</p> ^{6</sup> This affects heap-allocated locals too, but that's not a limitation of the language, only one of the impl.</p> </div>}// This example works today </span>let</span> iter = gen { </span> </span>let</span> cat_iter = cats.</span>into_iter</span>(); </span> </span>for</span> cat in cat_iter { </span> </span>yield</span> cat; </span> } </span>}; </span></code></pre> And to give a sense of what for example does not work, here is one of the samples Tmandry (T-Lang) collected. This creates an intermediate borrow, which results in the error: "Borrow may still be in use when gen</code> fn body yields" (playground</a>):</p> gen </span>fn </span>iter_set_rc</span><T: Clone>(</span>xs</span>: Rc<RefCell<HashSet<T>>>) ... { </span> </span>for</span> x in xs.</span>borrow</span>().</span>iter</span>() { </span> </span>yield</span> x.</span>clone</span>(); </span> } </span>} </span></code></pre> In order to enable examples like the last one to work, Rust needs to be able to express some form of "address-sensitive iterator". The obvious starting point would be to mint a new trait PinnedIterator</code> which changes the self-type of next</code> to take a Pin<&mut Self></code> rather than &mut self</code>:</p> trait </span>IntoIterator { </span> </span>type </span>Item; </span> </span>type </span>IntoIter: Iterator<Item = </span>Self::</span>Item>; </span> </span>fn </span>into_iter</span>(</span>self</span>) -> </span>Self::</span>IntoIter; </span>} </span> </span>trait </span>Iterator { </span> </span>type </span>Item; </span> </span>fn </span>next</span>(</span>self</span>: Pin<&</span>mut Self</span>>) -> Option<</span>Self::</span>Item>; </span>// ← `Pin<&mut Self>` </span> </span>fn </span>size_hint</span>(&</span>self</span>) -> (</span>usize</span>, Option<</span>usize</span>>) { .. } </span>} </span></code></pre> Enumerating all the problems of Pin</code> is worth its own blog post. But still, it seems important enough to point out that this definition has what Rust for Linux calls The Safe Pinned Initialization Problem</a>. IntoIterator::into_iter</code> cannot return a type that is address-sensitive at time of construction, instead address-sensitivity is something that can only be guaranteed at a later point once the type is pin!</code>ned in place.</p> At the start of this post I used the phrase: "(soft-)deprecation of the Iterator</code> trait". With that I was referring to one proposed idea to enable gen {}</code> to return a new trait Generator</code> with the same signature as our example. As well as some bridging impls for the purposes of compatibility. The core of the compat system would be as follows:</p> /// All `Iterator`s are `Generator`s </span>impl</span><I: IntoIterator> IntoGenerator </span>for </span>I { </span> </span>type </span>Item = I::Item; </span> </span>type </span>IntoGen = IteratorGenerator<</span>I::</span>IntoIter>; </span> </span>fn </span>into_gen</span>(</span>self</span>) -> </span>Self::</span>IntoGen { </span> IteratorGenerator(</span>self</span>.</span>into_iter</span>()) </span> } </span>} </span> </span>/// Only pinned `Generator`s are `Iterator`s </span>impl</span><G> Iterator </span>for </span>Pin<G> </span>where </span> G: DerefMut, </span> </span>G::</span>Target: Generator, </span>{ </span> </span>type </span>Item = <<G as Deref>::Target as Generator>::Item; </span> </span>fn </span>next</span>(&</span>mut </span>self</span>) -> Option<</span>Self::</span>Item> { </span> Generator::next(</span>self</span>.</span>as_mut</span>()) </span> } </span>} </span></code></pre> This creates a situation that I've been describing as "one-and-a-half-way compat", as opposed to the usual two-way-compat. And we need two-way-compat to not be a breaking change. This leads to a situation where changing a bound from taking Iterator</code> to Generator</code> is backwards-compatible. But changing an impl from returning an Iterator</code> to returning a Generator</code> is not backwards-compatible. The obvious solution then would be to migrate the entire ecosystem to take Generator</code> bounds everywhere. Coupled with gen {}</code> only ever returning Generator</code> and not Iterator</code>: that is a deprecation of Iterator</code> in all but name.</p> At first sight it might seem like we're being forced into deprecating Iterator</code> because of the limitations of Pin</code>. The obvious answer to that would be to solve the issues with Pin</code> by replacing it with something better</a>. But that creates a false dichotomy: there is nothing forcing us to make a decision on this today. As we established at the start of this section: a surprising amount of use cases already work without the need for address-sensitive iterators. And as we've seen throughout this post: address-sensitive iteration is far from the only feature that gen {}</code> blocks will not be able to support on day one.</p> Iterator Guaranteeing Destruct</h2> The current formulation of thread::scope</code> requires that the thread it’s called on remains blocked until all threads have been joined. This requires stepping into a closure and executing all code within that. Contrast this with something like FutureGroup</code></a> which logically owns computations and can be freely moved around. The values of the futures resolved within can in turn be yielded out. But unlike thread::scope</code> it can’t guarantee that all computations will complete, and so a parallel version of FutureGroup</code> can't mutably hold onto mutable borrows the way thread::scope</code> can.</p> // A usage example of `thread::scope`, </span>// the ability to spawn new threads </span>// is only available inside the closure. </span>thread::scope(\|</span>s</span>\| { </span> s.</span>spawn</span>(\|\| ..); </span> s.</span>spawn</span>(\|\| ..); </span> </span>// ← All threads are joined here. </span>}); </span> </span>// A usage example of `FutureGroup`, </span>// no closures required. </span>let mut</span> group = FutureGroup::new(); </span>group.</span>insert</span>(future::ready(</span>2</span>)); </span>group.</span>insert</span>(future::ready(</span>4</span>)); </span>group.</span>for_each</span>(\|_\| ()).await; </span></code></pre> If we want to write a type similar to FutureGroup</code> with the same guarantees as thread::scope</code>, we'd either need to guarantee that FutureGroup</code> can never be dropped or guarantee that FutureGroup</code>'s Drop</code> impl is always run. It turns out that it's rather impractical to have types that can't be dropped in a language where every function may panic. So the only real option here are to have types whose destructors are guaranteed to run.</p> The most plausible way we know of to do this would be by introducing a new auto trait Leak</code>, disallowing types from being passed to mem::forget</code>, Box::leak</code>, and so on. For more on the design, read Linear Types One-Pager</a>. Because Leak</code> is an auto-trait, we could compose it with the existing Iterator</code> and IntoIterator</code> traits, similar to Send</code> and Move</code>:</p> fn </span>linear_sink</span>(</span>iter</span>: impl IntoIterator<IntoIter: </span>?</span>Leak>) { .. } </span></code></pre> Async Iterator</h2> In Rust the async</code> keyword can transform imperative function bodies into state machines that can be manually advanced by calling the Future::poll</code> method. Under the hood this is done using what is called a coroutine transform</em>, which is the same transform we use to desugar gen {}</code> blocks with. But that's just the mechanics of it; the async</code> keyword in Rust also introduces two new capabilities</a>: ad-hoc concurrency and ad-hoc cancellation. Together these capabilities can be combined to create new control-flow operations, like Future::race</code></a> and Future::timeout</code>.</p> Async Functions in Traits were stabilized one year ago in Rust 1.75, enabling the use of async fn</code> in traits. Making the Iterator</code>trait work with async</code> is mostly a matter of adding an async</code> prefix to next</code>:</p> trait </span>IntoIterator { </span> </span>type </span>Item; </span> </span>type </span>IntoIter: Iterator<Item = </span>Self::</span>Item>; </span> </span>fn </span>into_iter</span>(</span>self</span>) -> </span>Self::</span>IntoIter; </span>} </span> </span>trait </span>Iterator { </span> </span>type </span>Item; </span> async </span>fn </span>next</span>(&</span>mut </span>self</span>) -> Option<</span>Self::</span>Item>; </span>// ← async </span> </span>fn </span>size_hint</span>(&</span>self</span>) -> (</span>usize</span>, Option<</span>usize</span>>) { .. } </span>} </span></code></pre> While the next</code> method here would be annotated with the async</code> keyword, the size_hint</code> method should probably not be. The reason for that is that it acts as a simple getter, and it really shouldn't be performing any asynchronous computation. It's also unclear whether into_iter</code> should be an async fn</code> or not. There is probably a pattern to be established here, and it might very well be.</p> An combination of iterator variants that's been of some interest recently is an address-sensitive async iterator. We could imagine writing an address-sensitive async iterator by making next</code> take self: Pin<&mut Self></code>:</p> trait </span>IntoIterator { </span> </span>type </span>Item; </span> </span>type </span>IntoIter: Iterator<Item = </span>Self::</span>Item>; </span> async </span>fn </span>into_iter</span>(</span>self</span>) -> </span>Self::</span>IntoIter; </span>} </span> </span>trait </span>Iterator { </span> </span>type </span>Item; </span> async </span>fn </span>next</span>(</span>self</span>: Pin<&</span>mut Self</span>>) -> Option<</span>Self::</span>Item>; </span>// ← async + `Pin<&mut Self>` </span> </span>fn </span>size_hint</span>(&</span>self</span>) -> (</span>usize</span>, Option<</span>usize</span>>) { .. } </span>} </span></code></pre> This signature is likely to confuse some people. async fn next</code> return an impl Future</code> which itself must be pinned prior to being polled. In this example we are separately requiring that Self</code> is also pinned. That is because "the state of the iterator" and "the state of the future" are not the same state. We intuitively understand this when working with non-async address-sensitive iterators: the locals created within the next</code> are not captured by the enclosing iterator, and are free to be stack-pinned for the duration of the call to next</code>. But when working with an asynchronous address-sensitive iterator, for some reason people seem to assume that all locals defined in fn next</code> now need to be owned by the iterator and not the future.</p> In the async Rust ecosystem there exists a popular variation of an async iterator trait called Stream</code>. Rather than keeping the state of the iterator (self</code>) and the next</code> function separate, it combines both into a single state. The trait has a single method poll_next</code> which acts as a mix between Future::poll</code> and Iterator::next</code>. With a provided convenience function async fn next</code> that is a thin wrapper around poll_next</code>.</p> trait </span>IntoStream { </span> </span>type </span>Item; </span> </span>type </span>IntoStream: Stream<Item = </span>Self::</span>Item>; </span> </span>fn </span>into_stream</span>(</span>self</span>) -> </span>Self::</span>IntoStream; </span>} </span> </span>pub trait </span>Stream { </span> </span>type </span>Item; </span> </span>fn </span>poll_next</span>( </span>// ← `fn poll_next` </span> </span>self</span>: Pin<&</span>mut Self</span>>, </span>// ← `Pin<&mut Self>` </span> </span>cx</span>: &</span>mut </span>Context<'_>, </span>// ← `task::Context` </span> ) -> Poll<Option<</span>Self::</span>Item>>; </span>// ← `task::Poll` </span> async </span>fn </span>next</span>(&</span>mut </span>self</span>) -> </span>Self::</span>Item </span>// ← `async` </span> </span>where </span> </span>Self</span>: Unpin </span>// ← `Self: Unpin` </span> { .. } </span> </span>fn </span>size_hint</span>(&</span>self</span>) -> (</span>usize</span>, Option<</span>usize</span>>) { ... } </span>} </span></code></pre> By combining both states into a single state, this trait violates one of the core tenets of async Rust's design: the ability to uniformly communicate cancellation by dropping futures. Here if the future by fn next</code> is dropped, that is a no-op and cancellation will not occur. This causes compositional async control-flow operators like Future::race</code> to not work despite compiling.</p> To instead cancel the current call to next</code> you are forced to either drop the entire stream, or use some bespoke method to cancel just the future's state. Cancellation in async Rust is infamous for being hard to get right, which is understandable when (among other things) core traits in the ecosystem do not correctly handle it.</p> Concurrent Iterator</h2> As we're approaching the end of our exposition here, let's talk about the most elaborate variations on Iterator</code>. First in line: the rayon</code> crate and the ParallelIterator</code> trait. Rayon provides what are called "parallel iterators" which process items concurrently rather than sequentially, using operating system threads. This tends to significantly improve throughput compared to sequential processing, but have the caveat that all consumed items must implement Send</code>. To see just how familiar parallel iterators can be: the following example looks almost identical to a sequential iterator except for the call to into_par_iter</code> instead of into_iter</code>.</p> use </span>rayon::prelude::; </span> </span>(</span>0</span>..</span>100</span>) </span> .</span>into_par_iter</span>() </span>// ← Instead of calling `into_iter`. </span> .</span>for_each</span>(\|</span>x</span>\| println!("</span>{:?}</span>", x)); </span></code></pre> The ParallelIterator</code> trait however comes as a pair with the Consumer</code> trait. It can be a little mind-boggling but the way Rayon works is that combinators can be chained to create a handler, which at the end of the chain is copied to each thread and used there to handle items. This is of course a simplified explanation; I'll defer to Rayon maintainers to provide a detailed explanation. To give you a sense how different these traits are from the regular Iterator</code>traits, here they are (simplified):</p> /// A consumer is effectively a generalized “fold” operation. </span>pub trait </span>Consumer<Item>: Send + Sized { </span> </span>/// The type of folder that this consumer can be converted into. </span> </span>type </span>Folder: Folder<Item, Result = </span>Self::</span>Result>; </span> </span>/// The type of reducer that is produced if this consumer is split. </span> </span>type </span>Reducer: Reducer<</span>Self::</span>Result>; </span> </span>/// The type of result that this consumer will ultimately produce. </span> </span>type </span>Result: Send; </span>} </span> </span>/// A type that can be iterated over in parallel </span>pub trait </span>IntoParallelIterator { </span> </span>/// What kind of iterator are we returning? </span> </span>type </span>Iter: ParallelIterator<Item = </span>Self::</span>Item>; </span> </span>/// What type of item are we yielding? </span> </span>type </span>Item: Send; </span> </span>/// Return a stateful parallel iterator. </span> </span>fn </span>into_par_iter</span>(</span>self</span>) -> </span>Self::</span>Iter; </span>} </span> </span>/// Parallel version of the standard iterator trait. </span>pub trait </span>ParallelIterator: Sized + Send { </span> </span>/// The type of item that this parallel iterator produces. </span> </span>type </span>Item: Send; </span> </span>/// Internal method used to define the behavior of this </span> </span>/// parallel iterator. You should not need to call this </span> </span>/// directly. </span> </span>fn </span>drive_unindexed</span><C>(</span>self</span>, </span>consumer</span>: C) -> </span>C::</span>Result </span> </span>where </span> C: UnindexedConsumer<</span>Self::</span>Item>; </span>} </span></code></pre> What matters most here is that using</em> the ParallelIterator</code> trait feels similar to a regular iterator. All you need to do is call into_par_iter</code> instead of into_iter</code> and you're off to the races. On the consuming side it seems like we should be able to author some variation of for..in</code> to consume parallel iterators. Rather than speculate about syntax, we can look at the signature of ParallelIterator::for_each</code> to see which guarantees this would need to make.</p> fn </span>for_each</span><F>(</span>self</span>, </span>f</span>: F) </span>where </span> F: Fn(</span>Self::</span>Item) + Sync + Send </span>{ .. } </span></code></pre> We can observe three changes here from the base iterator trait:</p> Self</code> no longer needs to be Sized</code>.</li> Somewhat predictably the closure F</code> needs to be thread-safe.</li> The closure F</code> needs to implement Fn</code> rather than FnMut</code> to prevent data races.</li> </ol> We can then infer that in the case of a parallel for..in</code> expression, the loop body would not be able to close over any mutable references. This is an addition to the existing limitation that loop bodies already can't express FnOnce</code> semantics and move values (e.g. "Warning: this value was moved in a previous iteration of the loop"</em>.)</p> An interesting combination of are "parallel iteration" and "async iteration". An interesting aspect of the async</code> keyword in Rust is that it allows for ad-hoc concurrent execution of futures without needing to rely on special syscalls or OS threads. This means that concurrency and parallelism can be detached from one another. While we haven't yet seen a "parallel async iterator" trait in the ecosystem, the futures-concurrency</code> crate does encode a "concurrent async iterator" ^{7</a></sup>. Just like ParallelIterator</code>, ConcurrentAsyncIterator</code> comes in a pair with a Consumer</code>trait.</p>} ^{7</sup> Technically this trait is called ConcurrentStream</code>, but there is little that is Stream</code>-dependent here. I called it that because it is compatible with the futures_core::Stream</code> trait since futures-concurrency</code> is intended to be a production crate.</p> </div>}/// Describes a type which can receive data. </span>pub trait </span>Consumer<Item, Fut> </span>where </span> Fut: Future<Output = Item>, </span>{ </span> </span>/// What is the type of the item we’re returning when completed? </span> </span>type </span>Output; </span> </span>/// Send an item down to the next step in the processing queue. </span> async </span>fn </span>send</span>(</span>self</span>: Pin<&</span>mut Self</span>>, </span>fut</span>: Fut) -> ConsumerState; </span> </span>/// Make progress on the consumer while doing something else. </span> async </span>fn </span>progress</span>(</span>self</span>: Pin<&</span>mut Self</span>>) -> ConsumerState; </span> </span>/// We have no more data left to send to the `Consumer`; </span> </span>/// wait for its output. </span> async </span>fn </span>flush</span>(</span>self</span>: Pin<&</span>mut Self</span>>) -> </span>Self::</span>Output; </span>} </span> </span>pub trait </span>IntoConcurrentAsyncIterator { </span> </span>type </span>Item; </span> </span>type </span>IntoConcurrentAsyncIter: ConcurrentAsyncIterator<Item = </span>Self::</span>Item>; </span> </span>fn </span>into_co_iter</span>(</span>self</span>) -> </span>Self::</span>IntoConcurrentAsyncIter; </span>} </span> </span>pub trait </span>ConcurrentAsyncIterator { </span> </span>type </span>Item; </span> </span>type </span>Future: Future<Output = </span>Self::</span>Item>; </span> </span> </span>/// Internal method used to define the behavior </span> </span>/// of this concurrent iterator. You should not </span> </span>/// need to call this directly. </span> async </span>fn </span>drive</span><C>(</span>self</span>, </span>consumer</span>: C) -> </span>C::</span>Output </span> </span>where </span> C: Consumer<</span>Self::</span>Item, </span>Self::</span>Future>; </span>} </span></code></pre> While ParallelIterator</code> and ConcurrentAsyncIterator</code> have similarities in both usage and design, they are different enough that we cant quite think as one being the async, non-thread-safe version of the other. Perhaps it is possible to bring both traits closer to one another, so that the only difference are a few strategically placed async</code> keywords, but more research is needed to validate whether that is possible.</p> Another interesting bit to point out here: concurrent iteration is also mutually exclusive with lending iteration. A lending iterator relies on yielded items having sequential lifetimes, while concurrent lifetimes rely on yielded items having overlapping lifetimes. Those are fundamentally incompatible concepts.</p> Conclusion</h2> And that's every iterator variant that I know of, bringing us to 17 different variations. If we take away the variants that we can express using subtraits (4 variants) and auto-traits (5 variants), we are left with 9 different variants. That's 9 variants, with 76 methods, and approximately 150 trait impls in the stdlib. That is a big API surface, and that's not even considering all the different combinations of iterators.</p> Iterator</code> is probably the single-most complex trait in the language. It is a junction in the language where every effect, auto-trait, and lifetime feature ends up intersecting. And unlike similar junctions like the Fn</code>-family of traits; the Iterator</code> trait is stable, broadly used, and has a lot of combinators. Meaning it both has a broad scope, and stringent backwards-compatibility guarantees we need to maintain.</p> At the same time Iterator</code> is also not that special either. It's a pretty easy trait to write by hand after all. The way I think of it is mainly as a canary for language-wide shortcomings. Iterator</code> is for example not unique in its requirement for stable addresses: we want to be able to guarantee this for arbitrary types and to be able to use this with arbitrary interfaces. I believe that the question to ask here should be: what is stopping us from using address-sensitive types with arbitrary types and arbitrary interfaces? If we can answer that, not only will we have an answer for Iterator</code> - we will have solved it for all other interfaces we did not consciously anticipate would want to interact with this 8</a></sup>.</p> ^{8</sup> This is adjacent to "known unknowns" vs "unknown unknowns" - we should not just cater to cases we can anticipate, but also to cases we cannot. And that requires analyzing patterns and thinking in systems.</p> </div> In this post I've done my best to show by-example which limitations the Iterator</code> trait has today, and how each variant can overcome those. And while I believe that we should try and lift those limitations over time, I don't think anyone is too keen on us minting 9 new variations on the core::iter</code> submodule. Nor the thousand-plus possible combinations of those submodules (yay combinatorics). The only feasible approach I see to navigating the problem space is by extending, rather than replacing, the Iterator</code> trait. Here is my current thinking for how we might be able to extend Iterator</code> to support all iterator variants:</p>} base trait</strong>: default, already supported</li> dyn-compatible</strong>: default, already supported</li> bounded</strong>: sub-trait, already supported</li> fused</strong>: sub-trait, already supported</li> thread-safe</strong>: auto-trait, already supported</li> seeking</strong>: sub-trait</li> compile-time</strong>: effect polymorphism (const</code>) 9</a></sup></li> lending</strong>: 'move</code> lifetime 10</a></sup></li> with return value</strong>: unsure 11</a></sup></li> with next argument</strong>: default value + optional/variadic arg</li> short-circuiting</strong>: effect polymorphism (try</code>)</li> address-sensitive</strong>: auto-trait</li> guaranteeing destruct</strong>: auto-trait</li> async</strong>: effect polymorphism (async</code>)</li> concurrent</strong>: new trait variant(s)</li> </ul> ^{9</sup> the const</code> effect itself is already polymorphic over the comptime effect, since const fn</code> means: "a function that can be executed either at comptime or runtime". Out of all the effect variants, const</code> is most likely to happen in the near-term.</p> </div>}^{10</sup> What we want to express is that we have an associated type which MAY take a lifetime, not MUST take a lifetime. That way we can pass a type by value where a type would otherwise be expected to be passed by-reference. This is different from both 'static</code> lifetimes and &own</code> references.</p> </div>}^11</sup>I tried answering how to add return values to the Iterator</code> trait a year ago in my post Iterator as an Alias</a>, but I came up short. As I mentioned earlier in the post: combining "return with value" and "may short-circuit with error" seems tricky. Maybe there is a combination here I'm missing / we can special-case something. But I haven't seen it yet.</p> </div> When evolving the language, I believe we entire job is to balance feature development with the management of complexity. If done right, over time the language should not only become more capable, but also simpler and easier to extend. To quote something TC (T-Lang) said in a recent conversation: "We should get better at getting better every year". 12</a></sup></p> As we think about how we wish to overcome the challenges presented by this post, it is my sincere hope that we will increasingly start thinking of ways to solve classes of problems that just happen to show up with Iterator</code> first. While at the same time looking for opportunities to ship features sooner, without blocking ourselves on supporting all of the use cases straight away.</p> ^{12</sup> I recently remarked to TC that I've started to think about governance and language evolution as being about the derivative of the language and project. Rather than measuring the direct outcomes, we measure the process that produces those outcomes. TC remarked that what we should really care about is the double derivative. Not only should we improve our outcomes over time; the processes that produce those outcomes should improve over time as well. Or put differently: we should get better at getting better every year! I love this quote and I wanted y'all to read it too.</p> </div>} Musings on iterator trait names Mon, 20 Jan 2025 00:00:00 +0000 At the end of my last post</a> I mentioned that one of the main issues with the IntoIterator</code> trait is that it’s kind of a pain to write. I wasn’t around when it was first introduced, but it’s not hard to tell that the original authors intended for Iterator</code> to be the primary interface with IntoIterator</code> being an additional convenience.</p> This didn’t quite turn out to be the case though, and it’s common practice to use IntoIterator</code> in both bounds and impls. In the Rust 2024 edition we’re changing Rust’s range type to implement IntoIterator</code> rather than Iterator</code>.</del> 1</a></sup> And for example in Swift the equivalent trait to IntoIterator</code> (Sequence</code>) is the primary interface used for iteration. With the interface equivalent to Iterator</code> (IteratorProtocol</code>) having a much harder to use name.</p> ^{1</sup> Thanks to Lukas Wirth for pointing out that the range type change didn't end up making the cut for the edition. It's been a couple of months since I checked, and it seems it was removed for this edition. My understanding is that this change is still desired, and might make it in for a future edition.</p> </div> So if not Iterator</code>, what name could we use? Well, recently I wrote a little library called Iterate</code> that tries to answer that question. Let me walk you through it.</p> Note: this post is intended to be a public exploration, not a concrete proposal. It's a starting point, asking: "... what if?" I'm a firm advocate for sharing ideas in public, especially if they're not yet fully fleshed out. There are a lot of good reasons to do that, but above all else: I think it's fun!</em></p>}verbs, nouns, and traits</h2> In Rust most interfaces use verbs as their names. To read bytes from a stream you use the trait named Read</code>. To write bytes you use Write</code>. To debug something you use Debug</code>. And to calculate numbers you can use Add</code>, Mul</code> (multiply), or Sub</code> (subtract). Most traits in the Rust stdlib are used to perform concrete operations with, and the convention is to use verbs for that.</p> The stdlib does have one particularly interesting pairing in the form of Hash</code> (verb) and Hasher</code> (noun). From the documentation: "Types implementing Hash</code> are able to be hashed with an instance of Hasher</code>." Or put differently: the trait Hash</code> represents the operation</em> and the trait Hasher</code> represents the state</em>.</p> /// A hashable type. </span>pub trait </span>Hash { </span> </span>fn </span>hash</span><H: Hasher>(&</span>self</span>, </span>state</span>: &</span>mut</span> H); </span>} </span> </span>/// Represents state that is changed while hashing data. </span>pub trait </span>Hasher { </span> </span>fn </span>finish</span>(&</span>self</span>) -> </span>u64</span>; </span> </span>fn </span>write</span>(&</span>mut </span>self</span>, </span>bytes</span>: &[</span>u8</span>]); </span>} </span></code></pre> A verb for iteration</h2> What the trait IntoIterator</code> really represents is: "An iterable type". And the trait Iterator</code> can be reasonably described as: "The state that is changed while iterating over items". The verb/noun split present in Hash</code>/Hasher</code> feels like it easily applies to iteration too.</p> If Iterator</code> is the noun that represents the iteration state, what is the verb that represents the capability? The obvious choice would be Iterate</code>. Which I think ends up working out somewhat nicely. To iterate over items you implement Iterate</code>, which provides you with a stateful Iterator</code>.</p> /// An iterable type. </span>pub trait </span>Iterate { </span> </span>type </span>Item; </span> </span>type </span>Iterator: Iterator<Item = </span>Self::</span>Item>; </span> </span>fn </span>iterate</span>(</span>self</span>) -> </span>Self::</span>Iterator; </span>} </span> </span>/// Represents state that is changed while iterating. </span>pub trait </span>Iterator { </span> </span>type </span>Item; </span> </span>fn </span>next</span>(&</span>mut </span>self</span>) -> Option<</span>Self::</span>Item>; </span>} </span></code></pre> With our goal to make IntoIterator</code> less jarring to use in interfaces, the name Iterate</code> doesn't seem half-bad. And it neatly follows the existing verb/noun split pairing we're already using elsewhere in the stdlib.</p> Collecting items</h2> People familiar with Rust's stdlib will be quick to note that Iterator</code> and IntoIterator</code> are not the only iteration traits in use. We also have FromIterator</code> that functions as the inverse of IntoIterator</code>. Where one exists to convert types to iterators, the other exists to convert iterators back to types. The latter is typically used via the Iterator::collect</code> function.</p> But IntoIterator</code> has a less-known but equally useful sibling: Extend</code>. Where IntoIterator</code> collects items into new instances of types, the Extend</code> trait is used to collect items into existing instances of types. It would feel pretty weird to rename IntoIterator</code> to Iterate</code>, but then keep FromIterator</code> as-is. What if instead of anchoring FromIterator</code> as a dual to IntoIterator</code>, we instead treated it as a sibling to Extend</code>. The obvious verb for this would be Collect</code>:</p> /// Creates a collection with the contents of an iterator. </span>pub trait </span>Collect<A>: Sized { </span> </span>fn </span>collect</span><T>(</span>iter</span>: T) -> </span>Self </span> </span>where </span> T: Iterate<Item = A>; </span>} </span></code></pre> It's interesting to note that the type T</code> in FromIterator</code> is bound by IntoIterator</code> rather than Iterator</code>. Being able to use T: Iterate</code> as a bound here definitely feels a little nicer. And speaking of nicer: this would also make the provided Iterator::collect</code> and Iterator::collect_into</code> methods feel a little better:</p> /// Represents state that is changed while iterating. </span>pub trait </span>Iterator { </span> </span>type </span>Item; </span> </span>fn </span>next</span>(&</span>mut </span>self</span>) -> Option<</span>Self::</span>Item>; </span> </span> </span>/// Create a collection with the contents of this iterator. </span> </span>fn </span>collect</span><C>(</span>self</span>) -> C </span> </span>where </span> C: Collect<</span>Self::</span>Item>, </span> </span>Self</span>: Sized, </span> </span> </span>/// Extend a collection with the contents of this iterator. </span> fn collect_into<E>(self, collection: &</span>mut</span> E) -> &</span>mut</span> E </span> </span>where </span> E: Extend<</span>Self::</span>Item>, </span> </span>Self</span>: Sized; </span>} </span></code></pre> I don't think this looks half-bad. And honestly: it might also be more consistent overall, as traits representing other effects don't have an equivalent to FromIterator</code>. The Future</code> trait only has IntoFuture</code>, and variations on that in the ecosystem like Race</code></a>. No longer having a trait called FromIterator</code> would help remove some confusion.</p> Async</h2> I guess we've broached the async topic now, so I guess we might as well keep going. We added the trait IntoFuture</code> to Rust in 2022 because we wanted an equivalent to IntoIterator</code> but for the async effect. You can find some motivating use cases for this in my async builders (2019)</a> post. We chose the name IntoFuture</code> because it matched the existing convention set by IntoIterator/Iterator</code>.</p> We already have Try</code> for fallibility, we just discussed using Iterate</code> for iteration, what would the verb-based trait name be for asynchrony? The obvious choice would be something like Await</code>, as that is the name of the operation:</p> trait </span>Await { </span> </span>type </span>Output; </span> </span>type </span>Future = Future<Output = </span>Self::</span>Output>; </span> </span>fn </span>await</span>(</span>self</span>) -> </span>Self::</span>Future; </span>} </span></code></pre> This however runs into one major limitation: await</code> is a reserved keyword, which means we can't use it as the name of the method. Which means I'm not actually sure what this trait should be called. With iterators we're lucky that we don't have any shortage of related words: loop</code>, iterate</code>, generate</code>, while</code>, sequence</code> and so on. With async we're a little shorter on words. If anyone has any good ideas for verbs to use here, I'd love to hear suggestions!</p> Conclusion</h2> TLDR: I really wouldn't mind Iterate</code> being the main interface name for iteration in Rust. That seems like it would be a step up from writing IntoIterator</code> in bounds everywhere. Just by changing a name, without the need for any special new language features.</p> Now for whether we should make this change:.. maybe? I honestly don't know. It's not just a matter of introducing a simple trait alias either: the method names and associated types are also different and we can't alias those. And I'm not particularly keen for Rust to start dabbling in additional trait hierarchies here either. Iteration is complex enough as it is, more super-traits are not going to make things any simpler here.</p> //! Renaming and aliasing the trait is not </span>//! enough, the method names and associated </span>//! type names would need to be aliased too. </span>pub trait </span>Iterate { .. } </span>pub trait </span>IntoIterator = Iterate; </span></code></pre> So I think the only way this rename would actually make sense to follow through on is if the process to make changes like these would make that change easy. I don't believe it is today, but I definitely believe we should want it to become easy in the future. It would be nice if we could freely rename traits, methods, and maybe even types across editions without causing any breakage.</p> Either way though: I had a lot of fun writing this post. If you want to try the Iterate</code> trait yourself today to get a better feel for it - check out the iterate-trait</code></a> crate. It has everything I've described in this post, as well as iterator combinators like map</code>. Probably don't use if for anything serious, but definitely go having fun with it.</p> The gen auto-trait problem Mon, 13 Jan 2025 00:00:00 +0000 One of the open questions surrounding the unstable gen {}</code> feature is whether it should return Iterator</code> or IntoIterator</code>. People have had a feeling there might be good reasons for it to return IntoIterator</code>, but couldn't necessarily articulate why</em>. Which is why it was included in the "unresolved questions" section on the gen blocks RFC.</p> Because I'd like to see gen {}</code> blocks stabilize sooner, I figured it would be worth spending some time looking into this question and see whether there are any reasons to pick one over the other. And I have found what I believe to be a fairly annoying issue with gen</code> returning Iterator </code> that I've started calling the gen auto-trait problem. In this post I'll walk through what this problem is, as well as how gen</code> returning IntoIterator</code> would prevent it. So without further ado, let's dive in!</p> Leaking auto-traits from gen bodies</h2> The issue I've found has to do with auto-trait impls on reified gen {}</code> instances. Take the thread::spawn</code> API: it takes a closure which returns a type T</code>. If you haven't seen its definition before, here it is:</p> pub fn </span>spawn</span><F, T>(</span>f</span>: F) -> JoinHandle<T> </span>where </span> F: FnOnce() -> T + Send + </span>'static</span>, </span> T: Send + </span>'static</span>, </span></code></pre> This signature states that the closure F</code> and return type T</code> must be Send</code>. But what it doesn't state is that all local values created inside the closure F</code> must be Send</code> too - which is common in the async world. Now let's create an iterator using a gen {}</code> block that we try and send across threads. We can create an iterator that yields a single u32:</p> let</span> iter = gen { </span> </span>yield</span> </span>12</span>u32</span>; </span>}; </span></code></pre> Now if we try and pass this to thread::spawn</code>, there are no problems. Our iterator implements + Send</code> and things will work as expected:</p> // ✅ Ok </span>thread::spawn(</span>move </span>\|\| { </span> </span>for</span> num in iter { </span> </span>println</span>("</span>{num}</span>"); </span> } </span>}).</span>unwrap</span>(); </span></code></pre> Now to show you the problem, let's try this again but this time with a gen {}</code> block that internally creates a std::rc::Rc</code>. This type is !Send</code>, which means that any type that holds it will also be !Send</code>. That means that iter</code> in our example here is now no longer thread-safe:</p> let</span> iter = gen { </span>// `-> impl Iterator + !Send` </span> </span>let</span> rc = Rc::new(...); </span> </span>yield</span> </span>12</span>u32</span>; </span> rc.</span>do_something</span>(); </span>}; </span></code></pre> That means if we try and send it across threads we'll get a compiler error:</p> // ❌ cannot send a `!Send` type across threads </span>thread::spawn(</span>move </span>\|\| { </span>// ← `iter` is moved </span> </span>for</span> num in iter { </span>// ← `iter` is used </span> </span>println</span>("</span>{num}</span>"); </span> } </span>}).</span>unwrap</span>(); </span></code></pre> And that right there is the problem. Even though iterators are lazy and the Rc</code> in our gen {}</code> block isn't actually constructed until iteration begins on the new thread, the generator block needs to reserve space for it internally and so it inherits the !Send</code> restriction. This leads to incompatibilities where locals defined entirely within gen {}</code> itself end up affecting the public-facing API. This ends up being very subtle and tricky to debug, unless you're already familiar with the generator desugaring.</p> Preventing the leakage</h2> Solving the gen's auto-trait problem is not that hard. What we want is for the !Send</code> fields in the generator to not show up in the generated type until we are ready to start iterating over it. That sounds a little scary, but in practice all we have to do is for gen {}</code> to start returning an impl IntoIterator</code> rather than an impl Iterator</code>. The actual Iterator</code> will still be !Send</code>, but our type IntoIterator</code> will be Send</code>:</p> let</span> iter = gen { </span>// `-> impl IntoIterator + Send` </span> </span>let</span> rc = Rc::new(...); </span> </span>yield</span> </span>12</span>u32</span>; </span> rc.</span>do_something</span>(); </span>}; </span></code></pre> Since our value iter</code> implements Send</code> we can now happily pass it across thread bounds. And our code continues to operate as expected, as for..in</code> operates on IntoIterator</code> which is implemented for all Iterator</code>:</p> // ✅ Ok </span>thread::spawn(</span>move </span>\|\| { </span>// ← `iter` is moved </span> </span>for</span> num in iter { </span>// ← `iter` is used </span> </span>println</span>("</span>{num}</span>"); </span> } </span>}).</span>unwrap</span>(); </span></code></pre> If you think about it, from a theoretical perspective this makes a lot of sense too. We can think of for..in</code> as our way of handling the iteration effect, which expects an IntoIterator</code>. gen {}</code> is the dual of that, used to create new instances of the iteration effect. It's not at all strange for it to return the same trait that for..in</code> expects. With the added bonus that it doesn't leak auto-traits from impl bodies to the type's signature.</p> What about other effects and auto-traits?</h2> This issue primarily affects effects which make use of the generator transform</em>. That is: iteration and async. In theory I believe that, yes, async {}</code> should probably return IntoFuture</code> rather than Future</code>. In WG Async we regularly see issues that have to do with auto-traits leaking from inside function bodies out into the type's signatures. If async</code> (2019) would have returned IntoFuture</code> (2022) rather than Future</code> (2019) that certainly seems like it could have helped. Though there would be a higher barrier to make that change today now that things are already stable.</p> On the trait side this doesn't just affect Send</code> either: it applies to all auto-traits, present and future. Though Send</code> is by far the most common trait people will experience issues with today, to a lesser extent this also already applies to Sync</code>. And in the future possibly also auto-traits like Freeze</code></a>, Leak</code></a>, and Move</code></a>. Though this shouldn't be the main motivator, preventing potential future issues is not worthless either.</p> Conclusion</h2> Admittedly the worst part of gen {}</code> returning IntoIterator</code> is that the name of the trait kind of sucks. IntoIterator</code> sounds auxiliary to Iterator</code>, and so it feels a little wrong to make it the thing we return from gen {}</code>. But on top of that: that's a lot of characters to write.</p> I wonder what would have happened if we'd taken an approach more similar to Swift. Swift has Sequence</code></a> which is like Rust's IntoIterator</code> and IteratorProtocol</code></a> which is like Rust's Iterator</code>. The primary interface people are expected to use is short and memorable. While the secondary interface isn't meant to be directly used, and so it has a much longer and less memorable name. As we're increasingly thinking of IntoIterator</code> as the primary interface for async iteration, maybe we'll want to revisit the trait naming scheme in the future.</p> In conclusion: it seems like a good idea to prevent auto-traits from leaking from gen bodies when reified. This is the kind of issue that if we can prevent we should prevent, as it can be both difficult to diagnose and annoying to work around. Having auto-trait leakage be a non-issue for generator blocks seems worthwhile.</p> What are temporal and spatial memory safety? Sun, 15 Dec 2024 00:00:00 +0000 The lovely folks working on security over at Google have recently been writing about "temporal (memory) safety"</a> and "spatial (memory) safety"</a>. When I first saw these terms it took me a minute to figure out what they meant, as searching for it online didn't yield immediate answers. So I figured it might be helpful to write it down for others to find:</p> Spatial memory safety:</strong> describes violations like out-of-bounds access. Say you have a vec of 10 items, it's undefined behavior if you try and read from the memory location of the non-existent 11th item. You can think of these as violations that have to do with memory regions</em> (space).</li> Temporal memory safety:</strong> describes violations like use-after-free. Say you have a type that has been de-initialized already ("dropped" in Rust), it's undefined behavior to then try and read from any of its fields. You can think of these as violations that have to do with the ordering of memory operations</em> (time).</li> </ul> My only qualm with these terms is that I've occasionally seen people drop the "memory" qualifier. I think this is part of what makes these terms confusing: there are other many more safety properties 1</a></sup> involving layout or ordering that can be modeled that don't have anything to do with memory safety. Skimming the entries on Temporal Logic</a> or TLA+</a> should make that clear enough 2</a></sup>.</p> ^1</sup>When modeling, the term "safety" refers to the absence of a defined negative property. That's in contrast to "liveness" which refers to the presence of a defined positive property. Read more</a>.</p> </div> ^{2</sup> If you prefer a concrete example: imagine two concurrent writers to the same TCP socket speaking HTTP. Without careful coordination between the two this will lead to issues involving ordering of operations</em> (temporal safety) that may lead to data loss, data corruption, and so on - but without ever involving involving any memory safety violations.</p> </div> My ask then is that when discussing spatial memory safety and temporal memory safety to refer to them as such in full. Don't try and shorten them by dropping the "memory" qualifier. Because when discussing formally modeled properties it certainly pays to be specific about what it is that you're trying to guarantee.</p>} Why `Pin` is a part of trait signatures (and why that's a problem) Tue, 15 Oct 2024 00:00:00 +0000 I've been wondering for a little while why the Future::poll</code> methods takes self: Pin<&mut Self></code> in its signature. I assumed there must have been a good reason, but when I asked my fellow WG Async members nobody seemed to know off hand why that was exactly. Or maybe they did, and I just had some trouble following. Either way, I think I've figured it out, and I want to spell it out for posterity so that others can follow too.</p> Why Pin</code> is part of method signatures</h2> Take for example a type MyType</code> and a trait MyTrait</code>. We can write an implementation of MyTrait</code> which is only available when MyType</code> is pinned:</p> trait </span>MyTrait { </span> </span>fn </span>my_method</span>(&</span>mut </span>self</span>) {} </span>} </span>struct </span>MyType; </span>impl</span><T> MyTrait </span>for </span>Pin<&</span>mut</span> MyType> {} </span></code></pre> Inside of functions we can even write bounds for this. Shout out here to Eric Holk who showed me that apparently the left-hand side of trait bounds can contain arbitrary types - not only generics or even types that are part of the function signature. I had no idea.</p> With that we can express that we're taking some type T</code> by-value, and once we pin that value it will implement MyTrait</code>:</p> fn </span>my_function</span><T>(</span>input</span>: T) </span>where </span> </span>for</span><</span>'a</span>> Pin<&</span>'a mut</span> T>: MyTrait, </span>{ </span> </span>let</span> pinned_input = pin!(input); </span>} </span></code></pre> Inside of MyTrait::my_method</code>, the type of self</code> will be &mut Pin<&mut Self></code>. That's not the same as the owned type Pin<&mut Self></code>, but luckily we can convert that into an owned type by calling Pin::as_mut</code></a>. The docs contain a big explainer why it's safe here to go from a mutable reference to an owned instance, which intuitively goes against Rust's ownership rules.</p> But what happens now if rather than writing a generic type T</code> with a where</code> clause, we instead want to use an impl trait in associated position (APIT). We might want to write something like this:</p> // how do we express those same bounds here? </span>fn </span>my_function</span><T>(</span>input</span>: impl ???) { </span> </span>let</span> pinned_input = pin!(input); </span>} </span></code></pre> But we have no way to express that exact bound. Unlike regular generics, APITs can't express the left-hand side of the bound (lvalue), they can only name the right-hand side (rvalue). This becomes even more pronounced when we try and use impl Trait</code> in return position (RPTIT).</p> Take for example a function that returns some type T</code>. Using concrete trait bounds we can express that this returns a type which when pinned implements MyTrait</code>:</p> fn </span>my_function</span><T>() -> T </span>where </span> </span>for</span><</span>'a</span>> Pin<&</span>'a mut</span> T>: MyTrait, </span>{ </span> MyType {} </span>} </span></code></pre> But if we then try and express that same function using RPTIT, we lose the ability to express that bound. The only solution to express an -> impl Trait</code> which exposes functionality when it's pinned, is to make Pin</code> directly part of the signature on the methods, and not implement the trait for a Pin<&mut Type></code>:</p> trait </span>MyTrait { </span> </span>fn </span>my_method</span>(</span>self</span>: Pin<&</span>mut Self</span>>) {} </span>// ← note the signature of self here </span>} </span>struct </span>MyType; </span>impl </span>MyTrait </span>for </span>MyType {} </span>// ← no longer implemented for `Pin<&mut MyType>` </span></code></pre> And now all of a sudden we can express -> impl MyTrait</code> whose methods can only be called when MyType</code> is pinned. With Unpin</code> being the opt-out for types where that is not the case.</p> fn </span>my_function</span>() -> impl MyTrait { </span>// Can be either pinned or not! </span> MyType {} </span>} </span></code></pre> Implications</h2> Concretely what this means is that if you want to have a trait that wants to work with pinned values and work with all language features like normal, you have to use self: Pin<&mut Self></code> part of the method signature. Maybe that's not a big deal for new traits, but it has implications for every existing trait in the stdlib.</p> Take for example the Iterator</code> trait. We can't just impl Iterator for Pin<&mut T></code> and expect RPTIT to work. Instead the expected route here seems like it should be to introduce a new trait PinnedIterator</code> that takes self: Pin<&mut T></code>. This is a backwards-incompatibility shared by all existing traits in the stdlib except for Future</code>, which already takes self: Pin<&mut Self></code>. That's a pretty big limitation, and something that's worth factoring into discussions about the viability of Pin</code> beyond Future</code>. For Iterator</code> it means we would want to mint at least the following variants:</p> // iterator </span>trait </span>Iterator { </span> </span>type </span>Item; </span> </span>fn </span>next</span>(&</span>mut </span>self</span>) -> Option<Item>; </span>} </span> </span>// address-sensitive iterator </span>trait </span>PinnedIterator { </span> </span>type </span>Item; </span> </span>fn </span>next</span>(</span>self</span>: Pin<&</span>mut</span> self>) -> Option<Item>; </span>} </span></code></pre> To run through some more implications of this: if we want to Rust users to be able to declare address-sensitive types in Rust, then the most likely path now is a duplication of the traits in the std::io</code> submodule taking a shape similar to this:</p> mod </span>io { </span> </span>pub trait </span>Read { ... } </span> </span>pub trait </span>PinnedRead { ... } </span> </span>pub trait </span>Write { ... } </span> </span>pub trait </span>PinnedWrite { ... } </span> </span>pub trait </span>Seek { ... } </span> </span>pub trait </span>PinnedSeek { ... } </span> </span>pub trait </span>BufRead { ... } </span> </span>pub trait </span>PinnedBufRead { ... } </span>} </span></code></pre> Pinning, like async</code> and try</code>, is a combinatorial property of traits which leads to an exponential amount of duplication. Lucky for us duplicating traits is not the only possible path we can take: some form of polymorphism for existing interfaces over Pin</code> seems possible too - as long as we're willing to change our formulation of it. Which was what lead me to formulate my design for the Move</code> auto-trait</a>, which is composable just like e.g. Send</code> and Sync</code>.</p> Conclusion</h2> I want to quickly shout out my fellow WG Async folks. We've been talking about this a bunch, and it's been helpful working through this. Even if it's taken me a couple of months to actually get around to posting this. Any mistakes in this post however are definitely my own.</p> In this post I mainly wanted to articulate why Future</code> takes self: Pin<&mut Self></code> rather than &mut self</code> and rely on impl Future for Pin<&mut T></code>. I think I've found a good reason for that, and it has again to do with the left- and right-hand side of bounds. For me this also confirms my hypothesis that any design for generalized self-referential types needs to be able to capture the following:</p> The ability to mark a type as being immovable</li> The ability to transition types from being movable to immovable</li> The ability to construct immovable types in-place</li> The ability to extend existing interfaces with immovability</li> The ability to describe self-referential lifetimes</li> The ability to safely initialize self-references without Option</code></li> </ol> This post specifically covered the fourth requirement: the ability to extend existing interfaces with immovability. Addressing that can take the form of duplication of interfaces (e.g. Iterator</code> vs PinnedIterator</code>), or composition via (auto-)traits (e.g. Move</code>). Other methods might be possible too, and I would encourage people with ideas to share them.</p> PoignardAzur has independently described</a> why and how pinned types need to be able to be constructed in-place. Their post showed examples of both the third and sixth requirements in the list. They introduced a form of emplacement via the -> pin Type</code> notation. This is similar to the more general -> super Type</code></a> notation I introduced in my post, which I adapted from Mara's super let</code> post</a>.</p> I hope this post helps at least partially clarify why Pin</code> needs to be part of interfaces. As well as helped spell out some of the logical consequences once we consider how it interacts with the stdlib's strict backwards-compatibility requirements. Because I believe we can and should do better than duplicating entire interfaces over the axis of immovability.</p> Solving my RSI issues Mon, 23 Sep 2024 00:00:00 +0000 For the past decade I've struggled with wrist injuries on and off. It's a common occupational injury for us keyboard users, and after my most recent bout this year I decided it was time to actually do something about it. Permanently. I like systemic solutions - they tickle a good part of my brain. And I figured I could apply some of that to my keyboard habits too.</p> In this post I want to share a little about what my journey to actually fix my wrist issues. Not to be prescriptive for others, but to be descriptive about what I'm doing. Perhaps it'll be useful for others, but mainly because it's fun for me to talk about. I've been making little ASCII cheat sheets for myself, and basically I just wanted to have an excuse to post them publicly.</p> Breaking up writing sessions</h2> The way I've dealt with my RSI issues for the past eight years is by taking repeated breaks. I've done this using the Time Out app</a> on macOS, and Workrave</a> on Windows and Linux. Both applications bring up a modal pop-up on a set schedule. I have it set to create a short 30-second break every 10 minutes or so, and a 10-minute break every 50 minutes.</p> While this does not make typing itself any less strenuous, it does break up some of the repetition - which ends up helping a lot. Taking breaks can be a little frustrating though, as it can break up flow. But it's better to have a lot of small breaks than longer-lasting injuries. Shout out to Jacob Groundwater for suggesting this back in 2016 - it's helped me a lot over the years.</p> Mobility</h2> Another thing I've been doing is mobility issues, specifically for my wrist. I worked with a physio from 2018 to 2020, and I have a stretch routine that helps with flexibility and mobility. I have to admit I've not been keeping that up, but as symptoms flare back up, it's something that I know I can apply when needed.</p> Where taking breaks helps break up the intensity of the strain, physical exercises help both with symptom relief and building resiliency. It's not perfect, but it's yet another thing I've found that helps me. I'm a little hesitant to provide instructions since I'm not a physio myself though, and it's definitely possible to hurt yourself if you don't do it right.</p> Choosing a keyboard layout</h2> I started taking regular breaks again in June this year, and it's not helped me as well as it did before. I assume it has something to do with being older now, and recovery just taking longer than it used to. Though I learned how to touch-type when I was in my early teens, I know that using QWERTY layouts aren't particularly good for you. It requires a lot of extra movement in fingers, which leads to a lot of extra strain.</p> Because I hope to continue to use computers in some capacity for the next fifty odd years, I figured re-learning something I only learned twenty years ago would be well-worth it in the long run. So time to bite the bullet, and learn a new layout. But which one?</p> I'm lucky that many of my friends are incredibly geeky</del>knowledgeable, and I figured one of them would probably have a good suggestion for what to learn. A quick check online suggested that Colemak Mod-DH</a> would probably be a good choice. So I checked with my friend Sy Brand, who thought it was a good layout - but they suggested something better: Canary</a>.</p> Canary is a fun little layout based on Colemak Mod-DH but optimizes for something called "rolls". In keyboard terminology a "roll" is a sequence of keys you can type with one hand. A layout which has "high rolls" means it's a layout that has many of those sequences. Here is what the layout looks like on a regular keyboard:</p> w l y p k z x o u ; [ ] \ </span> c r s t b f n e i a ' </span> j v d g q m h / , . </span></code></pre> Rolls in QWERTY are fairly uncommon. But with Canary a lot of words incorporate rolls. This might be easier to show rather than explain, so here are some words that incorporate rolls on a QWERTY layout: asimo</code>, werk</code>, wookie</code>. Do you notice how typing those feels kind of flowy? Now imagine nearly all words feel like that. Creating that feeling without causing strain is what the Canary layout was specifically designed for.</p> Learning a new keyboard layout</h2> Once I'd decided to switch to a new keyboard layout, there was a big "draw the rest of the owl" step to actually being able to use it. When I was a kid my mom paid for me to take after-school classes for a number of weeks to learn how to touch-type. That was 20 years ago though, and I somehow doubt there are in-person classes teaching the Canary layout. So what to do?</p> It turns out there are lots of great options now. I've been using keybr.com</a> which is free to use, supports various layouts in-browser, and provides different guided trainings to learn how to touch-type with new layouts. I've been using it daily for the past six weeks, and honestly it does everything I could hope for.</p> To complement keybr I've also started using ngram type</a> which allows you to train "N-grams". These are sequences of characters which are often grouped together. Those high rolls I was talking about earlier? N-grams are the sequences they refer to.</p> Finally I've also occasionally been using monkeytype</a> which is more focused on leaderboards and good looks. A lot of keyboard influencers (they exist) use this to show off how well they can type or how good their keyboard sounds. I haven't found it particularly useful to learn new layouts with, but it's been fun to play with!</p> On QWERTY I can comfortably type around 90 words per minute (wpm). When learning Canary it took me a few days to get to 10 wpm (unusable), two weeks to get to 20wpm (better, but still unusable) and about a month to get to 40 wpm (usable but slow). I spent about 10 total hours in keybr, averaging about 30 mins of training a day. I have no sense for whether that's fast or slow, but I was positively surprised that I could rebuild 20 years of muscle memory in a matter of weeks, with a fairly modest time investment.</p> Using vim movements on a new layout</h2> One thin I knew would be an issue and actually turned out to be an issue was editing code on a new layout. I use vim bindings in all of my editors, and vim motions are just part of me now. I asked Sy about how they switched over to a new layout when they did and deal with this, and well, they didn't. Good on them for not using vim bindings; sucks to be me.</p> This turned out to be a bit of a stumbling block: while I don't exclusively</em> write code, I do still write a lot of it. And that means that I need to be able to move around an editor. I mean, I know there are other vim movements and I use those too. But heck, not being able to use the base vim motions to navigate? Yeah, I wasn't going to give that up.</p> I looked into changing key bindings in VS Code, and tried figuring out what others had done - and there didn't seem to be any good solutions. Except one: Miryoku</a>.</p> Keyboard layers</h2> One of the issues with QWERTY is that you move around your fingers a lot, which causes strain over time. Miryoku is one attempt to define a layout which minimizes the amount of movement your fingers need to make. It's based on two principles:</p> Each finger should at most ever switch one key over</li> The strongest fingers (thumbs, index fingers) should do the most work</li> </ul> The way it achieves this is by defining an entire layout which can be accessed from the base position. The entire system is layer-based (modal), with each layer being accessible by pressing down a specific key with your thumb. Numbers, symbols, and navigation each get their own layer. And by putting the arrow keys on the position of the vim bindings, it makes it possible to restore vim-like navigation just by pressing down a single key. Here is what the neutral (base) layer looks like:</p> w l y p b z f o u ' ¹esc ⁴ret </span>c r s t g m n e i a ²spc ⁵bsp </span>q j v d k x h / , . ³tab ⁶del </span> ¹ ² ³ ⁴ ⁵ ⁶ </span></code></pre> But once we press and hold the space key (key 2) with our thumb, we engage the navigation layer and the arrow keys become available to us. It's not quite the same position as with vim - but you can configure it that way if you like too. Here's what it looks like by default:</p> - - - - - - - - - - </span>- - - - - - ← ↓ ↑ → </span>- - - - - - - - - - </span> ◼︎ - - - - - </span></code></pre> I thought this was neat and probably the right solution, but I wasn't sure how to engage the thumb keys on a regular keyboard. The space bar is in the way, and there is no thumb cluster for us to press. One solution is to shift your fingers one row up, and remap the keyboard to use the XQCBNM keys as our thumb keys. miryoku_kmonad</a> is a software package for macOS and Linux which allows you to do exactly that. But I tried and failed to get that to work reliably (or well: I lacked the patience to finish debugging it), so I went to find a different solution.</p> Hardware</h2> So far my plan was to keep using my existing hardware. I like my 60% Vortex Cypher keyboard, and I figured I'd start by switching keyboard layouts and see how far I could get. I tried that and got stuck after a while - so I figured I might as well commit to the bit and invest in hardware.</p> I ended up going picking up a pre-soldered Chocify keyboard</a> from Beekeeb</a>. This is a 36-key split keyboard which comes with Miryoku pre-installed. I got the bluetooth-based version with the screen which cost a little extra. I did make a mistake by choosing the "Choc Blue" switches which are different from Cherry MX blue switches, so I've got a new set of tactile switches coming in soon (Kaihl Sunset).</p> Custom keyboards are pretty involved, but well worth it. Some things I didn't know before I got started but had to learn on the way:</p> international shipping of batteries is difficult - if you need bespoke batteries for a hardware project in Europe, Amazon is actually one of the better options.</li> the Chocify board uses the nice!nano micro-controller. The way you flash this is by double-clicking a button that turns it into USB drive mode, and then you can just drag and drop new firmware on it from your computer</li> you can solder cables to connect them. Though you want to heat-shrink them after. I didn't actually end up doing this, but got help from my local coffee shop with this hah (if you're reading this: thanks Valdi!)</li> </ul> Picking up hardware was something I didn't want to do unless strictly necessary - but I think it was the right choice. I've recently also started experiencing shoulder pain after long typing sessions, and I've noticed that having a split ergo keyboard actually helps a lot with that too.</p> I'm fairly confident this little 36-key keyboard I have is going to become my new daily driver once I get the hang of it. And while that isn't the case yet, that moment is creeping up on me soon.</p> Learning a new keyboard layout (again)</h2> Oh before I close out, it turns out that the Canary keyboard layout comes in two variants: staggered and ortho. I didn't realize the two were different, so I just pressed "canary" on keybr and started learning. It turns out that was the staggered layout:</p> w l y p k z x o u ; [ ] \ </span> c r s t b f n e i a ' </span> j v d g q m h / , . </span></code></pre> On a 30% keyboard that layout ends up looking like this:</p> w l y p k z x o u ; </span>c r s t b f n e i a </span>j v d g q m h / , . </span></code></pre> The layout the Canary authors designed for keyboards like this is the "ortho" layout which looks like this. It's very similar, but not the same:</p> w l y p b z f o u ' </span>c r s t g m n e i a </span>q j v d k x h / , . </span></code></pre> Oops, I should have paid more attention before I got started. Though in all fairness: I did start learning Canary under the assumption I'd stick with my 60% staggered keyboard. So it makes sense I didn't immediately jump to the ortho layout. Though I probably should have thought to start learning the ortho layout as soon as I put in the hardware order - which was a few weeks ago now.</p> Conclusion</h2> Now that I have my keyboard, have the firmware flashed and batteries installed, this is where I'm currently at in my journey: re-learning to type on the Canary Ortho layout. I think this is fine, because when switching to the Miryoku layout I need to re-learn where all the symbols are, so it's not like I can hit the ground running on my new keyboard anyway.</p> I'm still waiting on my new key switches to arrive which should make typing on my new keyboard a lot easier (45g rather than 20g actuation force - MX blues have 50g actuation). But I can already tell that Canary + Miryoku + Chocify causes a lot less strain on my hands than QWERTY does.</p> I'm not going to say that people should follow what I'm doing. I don't know your situation, and maybe just getting an ergo QWERTY keyboard or taking more breaks might be enough for your needs. But for me? I've really enjoyed going down the rabbit hole of learning a new keyboard layout, getting a new keyboard, and picking up various things along the way.</p> So has all of this solved my RSI issues? No - I'm not yet daily driving my new keyboard, so I'm not yet experiencing the benefits. But I can tell the difference in strain I experience when I switch to the split keyboard and new keyboard layout. Now that nearly everything is in place to switch over, I can't wait to start daily driving it. I'm definitely taking the most involved path to solving this, but I think that this might actually be the answer to fully fix the source of my injuries.</p> This post was a little break from my usual writing about type systems and language design. If you'd like to learn more about ergonomic keyboard layouts, I really enjoyed this talk by Mattia Dal Ben: 34 keys is all you need: an ergonomic keyboard journey</a>.</p> </p> Here's what my keyboard practice looked like this morning. I think it's cute.</em></p> Further simplifying self-referential types for Rust Mon, 08 Jul 2024 00:00:00 +0000 In my last post</a> I discussed how we might be able to introduce ergonomic self-referential types (SRTs) to Rust, mostly by introducing features we know we want in some form anyway. The features listed were:</p> Some form of 'unsafe</code> and 'self</code> lifetimes.</li> A safe out-pointer notation for Rust (super let</code> / -> super Type</code>).</li> A way to introduce out-pointers without breaking backwards-compat.</li> A new Move</code> auto-trait that can be used to mark types as immovable (!Move</code>).</li> View types which make it possible to safely initialize self-referential types.</li> </ol> This post was received quite well, and I thought the discussion which followed was quite interesting. I learned about a number of things that I think would help refine the design further, and I thought it would be good to write it up.</p> Not all self-referential types are !Move</code></h2> Niko Matsakis pointed out that not all self-referential types are necessarily !Move</code>. For example: if the data being referenced is heap-allocated, then the type doesn't actually have to be !Move</code>. When writing protocol parsers, it's actually fairly common to read data into a heap-allocated type first. It seems likely that a fair number of self-referential types don't actually also need to be !Move</code> or any concept of Move</code> at all to function. Which also means we don't need some form of super let</code> / -> super Type</code> to construct types in-place.</p> If we just want to enable self-references for heap-allocated types, then all we need for that is a way to initialize them (view types) and an ability to describe the self-lifetimes ('unsafe</code> as a minimum). That should give us a good idea of what we can prioritize to begin enabling a limited form of self-references.</p> The 'self</code> lifetime is insufficient</h2> Speaking of lifetimes, Mattieum pointed out</a> that 'self</code> likely wasn't going to be enough. 'self</code> points to an entire struct, which ends up being too coarse to be practical. Instead we need to be able to point to individual fields to describe lifetimes.</p> Apparently Niko has also come up with a feature for this, in the form of lifetimes based on places</a>. Rather than having abstract lifetimes like the 'a</code> that we use to link to values with, it would be nicer if references just always had implicit, unique lifetime names. With access to that, we should rewrite the motivating example from our last post from being based on 'self</code>:</p> struct </span>GivePatsFuture { </span> </span>resume_from</span>: GivePatsState, </span> </span>data</span>: Option<String>, </span> </span>name</span>: Option<&</span>'self str</span>>, </span>// ← Note the `'self` lifetime </span>} </span></code></pre> To instead be based on paths:</p> struct </span>GiveManyPatsFuture { </span> </span>resume_from</span>: GivePatsState, </span> </span>first</span>: Option<String>, </span> </span>data</span>: Option<&</span>'self</span>.</span>first </span>str</span>>, </span>// ← Note the `'self.first` lifetime </span>} </span></code></pre> This might not seem like it's that important in this example; but once we introduce mutability things quickly spiral. And not introducing a magic 'self</code> in favor of always requiring 'self.field</code> seems like it would generally be better. And that requires having lifetimes which can be based on places, which seems like a great idea regardless.</p> Automatic referential stability</h2> Earlier in this post we established that we don't actually need to encode !Move</code> for self-referential types which store their values on the heap. That's not all self-referential types - but does describe a fair number of them. Now what if we didn't need to encode !Move</code> for almost the entire remaining set of self-referential types?</p> If that sounds like move constructors, you'd be right - but with a catch! Unlike the Relocate</code> trait I described in my last post, DoveOfHope noted</a> that we might not even need that to work. After all: if the compiler already knows that we're pointing to a field contained within the struct - can't the compiler make sure to update the pointers when we try and move the structure?</p> I was skeptical about the possibility of this until I read about place-based lifetimes. With that it seems like we would actually have enough granularity to know how to update which fields when they are moved. In terms of cost: that's just updating the value of a pointer on move - which is effectively free. And it would rid us almost entirely of needing to encode !Move</code>.</p> The only cases not covered by this would be 'unsafe</code> references or actual const T</code> / mut T</code> pointers to stack data. The compiler doesn't actually know what those point to, and so cannot update them on move. For that some form of Relocate</code> trait actually seems like it would be useful to have. But that's something that wouldn't need to be added straight away either.</p> Raw pointer operations and automatic referential stability</h2> This section was added after publication, on 2024-07-08.</em></p> While it should be possible for the compiler to guarantee that the codegen is correct for e.g. mem::swap</code></a>, we can't make those same guarantees for raw pointer operations such as ptr::swap</code></a>. And because existing structures may be freely using those operations internally, that means on-stack self-referential types can't just be made to work without any caveats the way we can for on-heap SRTs. That's indeed a problem, and I want to thank The_8472</a> for pointing this out.</p> I was really hoping to be able to avoid extra bounds, so that on-stack SRTs could match the experience of on-heap SRTs. But that doesn't seem possible, so perhaps some minimum amount of bounds which we can flip to being set by default (like Sized</code>) over an edition might be enough to do the trick here. I'm currently thinking of something like:</p> Introduce a new auto marker trait Transfer</code> to complement Relocate</code>, as a dual to Rust's Destruct</code></a> / Drop</code></a> system. Transfer</code> is the name of the bound people would use, Relocate</code> provides the hooks to extend the Transfer</code> system.</li> All types with 'self</code> lifetimes automatically implement Transfer</code>.</li> Only bounds which include + Transfer</code> can take impl Transfer</code> types.</li> All relevant raw pointer move operations must uphold additional safety invariants of what to do with impl Transfer</code> types.</li> We gradually update the stdlib to support + Transfer</code> in all bounds.</li> Over some edition we make opt-out rather than opt-in (T: Transfer</code> → T: ?Transfer</code>).</li> </ol> auto </span>trait </span>Transfer {} </span> </span>trait </span>Relocate { ... } </span></code></pre> I was really hoping we could avoid something like this. And it does put into question whether this is actually simpler than immovable types. But the_8472 is quite right that this is an issue, and so we need to address it. Luckily we've already done something like this before with const</code>. And I don't think that this is something we can generalize. I'll write more about this at some later date.</p> Relocate</code> should probably take &own self</code></h2> Now, even if we don't expect people to need to write their own pointer update logic, basically like ever, it's still something that should be provided. And when we do, we should encode it correctly. Nadrieril very helpfully pointed out that the &mut self</code> bound on the Relocate</code> trait might not actually be what we want - because we're not just borrowing a value - we actually want to destruct it. Instead they informed me about the work done towards &own</code>, which would give access to something called: "owned references".</p> Daniel Henry-Mantilla is the author of the stackbox</code> crate</a> as well as the main person responsible for the lifetime extension system behind the pin!</code> macro in the stdlib. A while back he shared a very helpful writeup about &own</code></a>. The core of the idea is that we should decouple the concepts of: "Where is the data concretely stored?"</em> from: "Who logically owns the data?"</em> Resulting in the idea of having a reference which doesn't just provide temporary</em> unique access - but can take permanent</em> unique access. In his post, Daniel helpfully provides the following table:</p> </th> Semantics for T</code></th> For the backing allocation</th></tr></thead> &T</code></td> Shared access</td> Borrowed</td></tr> &mut T</code></td> Exclusive access</td> Borrowed</td></tr> &own T</code></td> Owned access (drop responsibility)</td> Borrowed</td></tr> </tbody></table> Applying this to our post, we would use this to change the trait Relocate</code> from taking &mut self</code>, which temporarily takes exclusive access of a type - but can't actually drop the type:</p> trait </span>Relocate { </span> </span>fn </span>relocate</span>(&</span>mut </span>self</span>) -> super </span>Self</span>; </span>} </span></code></pre> To instead take &own</code>, which takes permanent exclusive access of a type, and can actually drop the type:</p> trait </span>Relocate { </span> </span>fn </span>relocate</span>(&own </span>self</span>) -> super </span>Self</span>; </span>} </span></code></pre> edit 2024-07-08: This example was added later</em>. To explain what &own</code> solves, let's take a look at the Relocate</code> example impl from our last post. In it we say the following:</p> We're making one sketchy assumption here: we need to be able to take the owned data from self, without running into the issue where the data can't be moved because it is already borrowed from self.</p> </blockquote> struct </span>Cat { </span> </span>data</span>: String, </span> </span>name</span>: &</span>'self str</span>, </span>} </span>impl </span>Cat { </span> </span>fn </span>new</span>(</span>data</span>: String) -> super </span>Self </span>{ ... } </span>} </span>impl </span>Relocate </span>for </span>Cat { </span> </span>fn </span>relocate</span>(&</span>mut </span>self</span>) -> super </span>Self </span>{ </span> </span>let mut</span> data = String::new(); </span>// ← dummy type, does not allocate </span> mem::swap(&</span>mut </span>self</span>.data, &</span>mut</span> data); </span>// ← take owned data </span> </span>super let</span> cat = Cat { data }; </span>// ← construct new instance </span> cat.name = cat.data.</span>split</span>(' ').</span>next</span>().</span>unwrap</span>(); </span>// ← create self-ref </span> cat </span>// ← return new instance </span> } </span>} </span></code></pre> What &own</code> gives us is a way to correctly encode the semantics here. Because the type is not moved we can't actually move by-value. But logically we do still want to claim unique ownership of the value so we can destruct the type and move individual fields. That's kind of the way moving Box</code> by-value works too, but rather than having the allocation on the heap, the allocation can be anywhere. With that we could rewrite the rather sketchy mem::swap</code> code above into more normal-looking destructuring + initialization instead:</p> struct </span>Cat { </span> </span>data</span>: String, </span> </span>name</span>: &</span>'self str</span>, </span>} </span>impl </span>Cat { </span> </span>fn </span>new</span>(</span>data</span>: String) -> super </span>Self </span>{ ... } </span>} </span>impl </span>Relocate </span>for </span>Cat { </span> </span>fn </span>relocate</span>(&</span>mut </span>self</span>) -> super </span>Self </span>{ </span> </span>let Self </span>{ data, .. } = </span>self</span>; </span>// ← destruct `self` </span> </span>super let</span> cat = Cat { data }; </span>// ← construct new instance </span> cat.name = cat.data.</span>split</span>(' ').</span>next</span>().</span>unwrap</span>(); </span>// ← create self-ref </span> cat </span>// ← return new instance </span> } </span>} </span></code></pre> Now, because this actually does need to construct types in fixed memory locations this trait would need to have some form of -> super Self</code> syntax. After all: this would be the one place where it would still be needed. For anyone interested in keeping up to date with &own</code>, here is the Rust issue for it</a> (which also happens to have been filed by Niko).</p> Motivating example, reworked again</h2> With this in mind, we can rework the motivating example from the last post once again. To refresh everyone's memory, this is the high-level async/.await</code>-based Rust code we would:</p> async </span>fn </span>give_pats</span>() { </span> </span>let</span> data = "</span>chashu tuna</span>".</span>to_string</span>(); </span> </span>let</span> name = data.</span>split</span>(' ').</span>take</span>().</span>unwrap</span>(); </span> </span>pat_cat</span>(&name).await; </span> println!("</span>patted </span>{name}</span>"); </span>} </span> </span>async </span>fn </span>main</span>() { </span> </span>give_pats</span>().await; </span>} </span></code></pre> And using the updates in this post we can proceed to desugar it. This time around without the need for any reference to Move</code> or in-place construction, thanks to path-based lifetimes and the compiler automatically preserving referential stability:</p> enum </span>GivePatsState { </span> Created, </span> Suspend1, </span> Complete, </span>} </span> </span>struct </span>GivePatsFuture { </span> </span>resume_from</span>: GivePatsState, </span> </span>data</span>: Option<String>, </span> </span>name</span>: Option<&</span>'self</span>.</span>data </span>str</span>>, </span>// ← Note the `'self.data` lifetime </span>} </span>impl </span>GivePatsFuture { </span>// ← No in-place construction needed </span> </span>fn </span>new</span>() -> </span>Self </span>{ </span> </span>Self </span>{ </span> resume_from: GivePatsState::Created, </span> data: None, </span> name: None </span> } </span> } </span>} </span>impl </span>Future </span>for </span>GivePatsFuture { </span> </span>type </span>Output = (); </span> </span>fn </span>poll</span>(&</span>mut </span>self</span>, </span>cx</span>: &</span>mut </span>Context<'_>) </span>// ← No `Pin` needed </span> -> Poll<</span>Self::</span>Output> { ... } </span>} </span></code></pre> This is significantly simpler than what we had before in its definition. And even the actual desugaring of the invocation ends up being simpler: no longer needing the intermediate IntoFuture</code> constructor to guarantee in-place construction.</p> let</span> into_future = GivePatsFuture::new(); </span>let mut</span> future = into_future.</span>into_future</span>(); </span>// ← No `pin!` needed </span>loop </span>{ </span> </span>match</span> future.</span>poll</span>(&</span>mut</span> current_context) { </span> Poll::Ready(ready) => </span>break</span> ready, </span> Poll::Pending => </span>yield</span> Poll::Pending, </span> } </span>} </span></code></pre> All that's needed for this is for the compiler to update the addresses of self-pointers on move. That's a little bit of extra codegen whenever a value is moved - rather than just a bitwise copy, it also needs to update pointer values. But it seems quite doable to implement, should actually perform really well, and most importantly: users would rarely if ever need to think about it. Writing &'self.field</code> would always just work.</p> When immovable types are still needed</h2> I don't want to discount the idea of immovable types entirely though. There are definitely benefits to having types which cannot be moved. Especially when working with FFI structures that require immovability. Or some high-performance data structures which use lots of self-references to stack-based data which would be too expensive to update. Use cases certainly exist, but they're going to be fairly niche. For example: Rust for Linux uses immovable types for their intrusive linked lists - and I think those probably need some form of immovability to actually work.</p> However, if the compiler doesn't require immovable types to provide self-references, then immovable types suddenly go from being load-bearing to instead becoming something closer to an optimization. It's likely still worth adding them since they certainly are more efficient. But if we do it right, adding immovable types will be backwards-compatible, and would be something we can introduce later as an optimization.</p> When it comes to whether async {}</code> should return impl Future</code> or impl IntoFuture</code>: I think the answer really should be impl IntoFuture</code>. In the 2024 edition we're changing range syntax (0..12</code>) from returning Iterator</code> to returning IntoIterator</code></a>. This matches Swift's behavior, where 0..12</code> returns Sequence</code></a> rather than IteratorProtocol</code></a>. I think this is a good indication that async {}</code> and gen {}</code> probably also should return impl Into</code> traits rather than their respective traits.</p> Conclusion</h2> I like it when something I write is discussed and I end up picking up on other relevant work. I think to enable self-referential types I'm now definitely favoring a form of built-in pointer updates as part of the language over immovable types (edit: maybe I spoke too soon</a>). However, if we do want immovable types - I think my last post provides a coherent and user-friendly design to get us there.</p> There are a fairly large number of dependencies if we want to implement a complete story for self-referential types. Luckily we can implement features one at a time, enabling increasingly more expressive forms of self-referential types. In terms of importance: some form of 'unsafe</code> seems like a good starting point. Followed by place-based lifetimes. View types seem useful, but aren't in the critical path since we can work around phased initialization using an option dance. Here's a graph of all features and their dependencies.</p> </p> Breaking down the features like has actually further reinforced my perception that this all seems quite doable. 'unsafe</code> doesn't seem like it's that far out. And Niko has sounded somewhat serious about path-based lifetimes and view types. We'll have to see how quickly those will actually end up being developed in practice - but having laid this all out like this, I'm feeling somewhat optimistic!</p> Ergonomic Self-Referential Types for Rust Mon, 01 Jul 2024 00:00:00 +0000 I've been thinking a little about self-referential types recently, and while it's technically possible</em> to write them today using Pin</code> (limits apply), they're not at all convenient. So what would it take to make it convenient? Well, as far as I can tell there are four components involved in making it work:</p> The ability to write 'self</code> lifetimes.</li> The ability to construct types from functions in fixed memory locations.</li> A way to mark types as "immovable" in the type system.</li> The ability to safely initialize self-references in structs without going through an option-dance.</li> </ol> It's only once we have all four of these components that writing self-referential types can become accessible to most regular Rust programmers. And that seems important, because as we've seen with async {}</code> and Future</code>: once you start writing sufficiently complex state machines, being able to track references into data becomes incredibly useful.</p> Speaking of async</code> and Future</code>: in this post we'll be using that as a motivating example for how these features can work together. Because if it seems realistic that we can make that a case as complex as that work, other simpler cases should probably work too.</p> Oh and before we dive in, I want to give a massive shout-out to Eric Holk. We've spent several hours working through the type-system implications of !Move</code> together, and worked through a number of edge cases and issues. I can't be solely credited for the ideas in this post. However any mistakes in this post are mine, and I’m not claiming to speak for the both of us.</p> Disclaimer: This post is not a fully-formed design. It is an early exploration of how several features could work together to solve a broader problem. My goal is primarily to narrow down the design space to a tangible list of features which can be progressively implemented, and share it with the broader Rust community for feedback. I'm not on the lang team, nor do I speak for the lang team.</em></p> Motivating example</h2> Let's take async {}</code> and Future</code> as our examples here. When we borrow local variables in an async {}</code> block across .await</code> points, the resulting state machine will store both the concrete value and a reference to that value in the same state machine struct. That state machine is what we call self-referential</em>, because it has a reference</em> which points to something in self</em>. And because references are pointers to concrete memory addresses, there are challenges around ensuring they are never invalidated as that would result in undefined behavior. Let's look at an example async function:</p> async </span>fn </span>give_pats</span>() { </span> </span>let</span> data = "</span>chashu tuna</span>".</span>to_string</span>(); </span>// ← Owned value declared </span> </span>let</span> name = data.</span>split</span>(' ').</span>next</span>().</span>unwrap</span>(); </span>// ← Obtain a reference </span> </span>pat_cat</span>(&name).await; </span>// ← `.await` point here </span> println!("</span>patted </span>{name}</span>"); </span>// ← Reference used here </span>} </span> </span>async </span>fn </span>main</span>() { </span> </span>give_pats</span>().await; </span>// Calls the `give_pats` function. </span>} </span></code></pre> This is a pretty simple program, but the idea should come across well enough: we declare an owned value in-line, we call an .await</code> function, and later on we reference the owned value again. This keeps a reference live across an .await</code> point, and that requires self-referential types. We can desugar this to a future state machine like so</a>:</p> enum </span>GivePatsState { </span> Created, </span>// ← Marks our future has been created </span> Suspend1, </span>// ← Marks the first `.await` point </span> Complete, </span>// ← Marks the future is now done </span>} </span> </span>struct </span>GivePatsFuture { </span> </span>resume_from</span>: GivePatsState, </span> </span>data</span>: Option<String>, </span> </span>name</span>: Option<&</span>str</span>>, </span>// ← Note the lack of a lifetime here </span>} </span> </span>impl </span>GivePatsFuture { </span> </span>fn </span>new</span>() -> </span>Self </span>{ </span> </span>Self </span>{ </span> resume_from: GivePatsState::Created, </span> data: None, </span> name: None, </span> } </span> } </span>} </span> </span>impl </span>Future </span>for </span>GivePatsFuture { </span> </span>type </span>Output = (); </span> </span>fn </span>poll</span>(</span>self</span>: Pin<&</span>mut Self</span>>, </span>cx</span>: &</span>mut </span>Context<'_>) -> Poll<</span>Self::</span>Output> { ... } </span>} </span></code></pre> The lifetime of GivePatsFuture::name</code> is unknown, mainly because we can't name it. And because the desugaring happens in the compiler, it doesn't have to name the lifetime either. We'll talk more about that later in this post. Because this generates a self-referential state machine, this future will need to be fixed in-place using Pin</code> first. Once pinned, the Future::poll</code> method can be called in a loop until the future yields Ready</code>. The desugaring for that will look something like this</a>:</p> let mut</span> future = IntoFuture::into_future(GivePatsFuture::new()); </span>let mut</span> pinned = </span>unsafe </span>{ Pin::new_unchecked(&</span>mut</span> future) }; </span>loop </span>{ </span> </span>match</span> pinned.</span>poll</span>(&</span>mut</span> current_context) { </span> Poll::Ready(ready) => </span>break</span> ready, </span> Poll::Pending => </span>yield</span> Poll::Pending, </span> } </span>} </span></code></pre> And finally, just for reference, here is what the traits we're using look like today. The main bit that's interesting here for the purpose of this post is that Future</code> takes a Pin<&mut Self></code>, which we'll be explaining how it can be replaced with a simpler system throughout the remainder of this post.</p> pub trait </span>IntoFuture { </span> </span>type </span>Output; </span> </span>type </span>IntoFuture: Future<Output = </span>Self::</span>Output>; </span> </span>fn </span>into_future</span>(</span>self</span>) -> </span>Self::</span>IntoFuture; </span>} </span> </span>pub trait </span>Future { </span> </span>type </span>Output; </span> </span>fn </span>poll</span>(</span>self</span>: Pin<&</span>mut Self</span>>, </span>cx</span>: &</span>mut </span>Context<'_>) -> Poll<</span>Self::</span>Output>; </span>} </span></code></pre> Now that we've taken a look at how self-referential futures are desugared by the compiler today, let's take a look at how we can incrementally replace it with a safe, user-constructible system.</p> Self-referential lifetimes</h2> In our motivating example we showed the GivePatsFuture</code> which has the name</code> field that points at the data</code> field. It's a clearly a reference, but it does not carry any lifetime:</p> struct </span>GivePatsFuture { </span> </span>resume_from</span>: GivePatsState, </span> </span>data</span>: Option<String>, </span> </span>name</span>: Option<&</span>str</span>>, </span>// ← Note the lack of a lifetime </span>} </span></code></pre> The reason this doesn't have a lifetime is not inherent</em>, it's because we can't actually name the lifetime here. It's not 'static</code> because it isn't valid for the remainder of the program. In the compiler today I believe we can just omit the lifetime because the codegen happens after the lifetimes have already been checked. But say we wanted to write this by hand today as-is; we would need a concept for an "unchecked lifetime", something like this:</p> struct </span>GivePatsFuture { </span> </span>resume_from</span>: GivePatsState, </span> </span>data</span>: Option<String>, </span> </span>name</span>: Option<&</span>'unsafe str</span>>, </span>// ← Unchecked lifetime </span>} </span></code></pre> Being able to write a lifetime that isn't checked by the compiler would be the first-click stop to enable writing self-referential structs by hand. It would just require a lot of unsafe</code> and anxiety to get right. But at least it would be possible. I believe folks on T-compiler are already working on adding this, which seems like a great idea.</p> But even better would be if we could describe checked</em> lifetimes here. What we actually want to write here is a lifetime which is valid for the duration of the value - and it would always be guaranteed to be valid. Adding this lifetime would come with additional constraints we'll get into later in this post (e.g. the type wouldn't be able to move), but what we really want is to be able to write something like this:</p> struct </span>GivePatsFuture { </span> </span>resume_from</span>: GivePatsState, </span> </span>data</span>: Option<String>, </span> </span>name</span>: Option<&</span>'self str</span>>, </span>// ← Valid for the duration of `Self` </span>} </span></code></pre> To sidetrack slightly; adding a named 'self</code> lifetime could also allow us to remove the where Self: 'a</code> boilerplate when using lifetimes in generic associated types. If you've ever worked with lifetimes in associated types, then you'll likely have run into the "missing required bounds" error. Niko suggested in this issue</a> using 'self</code> as one possible solution for the bounds boilerplate. I think its meaning would be slightly different than when used with self-references? But I don't believe using this would be ambiguous either. And all in all I think it looks rather neat:</p> // A lending iterator trait as we have to write it today: </span>trait </span>LendingIterator { </span> </span>type </span>Item<</span>'a</span>> </span> </span>where </span> </span>Self</span>: </span>'a</span>; </span> </span>fn </span>next</span>(&</span>mut </span>self</span>) -> </span>Self::</span>Item<'_>; </span>} </span> </span>// A lending iterator trait if we had `'self`: </span>trait </span>LendingIterator { </span> </span>type </span>Item<</span>'self</span>>; </span> </span>fn </span>next</span>(&</span>mut </span>self</span>) -> </span>Self::</span>Item<'_>; </span>} </span></code></pre> Constructing types in-place</h2> In order for 'self</code> to be valid, we have to promise our value won't move in memory. And before we can promise a value won't move, we have to first construct it somewhere we can be sure it can stay and not be moved further. Looking at our motivating example, the way we achieve this using Pin</code> is a little goofy. Here's that same example again:</p> impl </span>GivePatsFuture { </span> </span>fn </span>new</span>() -> </span>Self </span>{ </span> </span>Self </span>{ </span> resume_from: GivePatsState::Created, </span> data: None, </span> name: None, </span> } </span> } </span>} </span> </span>async </span>fn </span>main</span>() { </span> </span>let mut</span> future = IntoFuture::into_future(GivePatsFuture::new()); </span> </span>let mut</span> pinned = </span>unsafe </span>{ Pin::new_unchecked(&</span>mut</span> future) }; </span> </span>loop </span>{ </span> </span>match</span> pinned.</span>poll</span>(&</span>mut</span> current_context) { </span> Poll::Ready(ready) => </span>break</span> ready, </span> Poll::Pending => </span>yield</span> Poll::Pending, </span> } </span> } </span>} </span></code></pre> In this example we're seeing GivePatsFuture</code> being constructed inside of the new</code> function, be moved out of that, and only then pinned in-place using Pin::new_unchecked</code>. Even if GivePatsFuture: !Unpin</code>, the Unpin</code> trait only affects types once they are held inside of a Pin</code> structure. And we can't just return Pin</code> from new</code>, because the function's stack frames are discarded the moment the function returns.</p> It would be better if we enabled types to describe how they can construct themselves in-place. That means no more external Pin::new_unchecked</code> calls; but exclusively internally provided constructors. This enables us to make self-referential types entirely self-contained, with internally provided constructors replacing the external pin dance. Here's how we could rewrite GivePatsFuture::new</code> to use an internal constructor instead:</p> impl </span>GivePatsFuture { </span> </span>fn </span>new</span>(</span>slot</span>: Pin<&</span>mut </span>MaybeUninit<</span>Self</span>>>) { </span> </span>let</span> slot = </span>unsafe </span>{ slot.</span>get_unchecked_mut</span>() }; </span> </span>let</span> this: </span>mut Self </span>= slot.</span>as_mut_ptr</span>(); </span> </span>unsafe </span>{ </span> addr_of_mut!((this).resume_from).</span>write</span>(GivePatsState::Created); </span> addr_of_mut!((this).data).</span>write</span>(None); </span> addr_of_mut!((this).name).</span>write</span>(None); </span> }; </span> } </span>} </span></code></pre> If you don't like this: that's understandable. I don't think anyone does. But bear with me; we're on a little didactic sausage-making journey here together. I'm sorry about the sights; let's quickly move on.</p> In a recent blog post</a> I posited we might be able to think of parameters like these as "spicy return". James Munns pointed out that in C++ this feature has a name: out-pointers. And Jack Huey made an interesting connection between this and the super let</code> design</a>. Just so we don't have to look at a pile of unsafe</code> code again, let's pretend we can combine these into something coherent:</p> impl </span>GivePatsFuture { </span> </span>fn </span>new</span>() -> super Pin<&</span>'super mut Self</span>> { </span> pin!(</span>Self </span>{ </span>// just pretend this works </span> resume_from: GivePatsState::Created, </span> data: None, </span> name: None, </span> }) </span> } </span>} </span></code></pre> I know, I know - I'm showing syntax here. I'm personally meh</em> about the way it looks and we'll talk more later about how we can improve this, but I hope we can all agree that the function body itself is approximately 400% more legible than the pile of unsafe</code> we were working with earlier. We'll also get to how we can entirely remove Pin</code> from the signature, so please don't get too hung up on that either.</p> I'm asking you to play along here for a second, and pretend we might be able to do something like this for now, so we can get to the part later on this post where we can actually fix it. In maybe state this more clearly: this post is less about proposing concrete designs for a problem, but more about how we can tease apart the problem of "immovable types" into separate features we can tackle independently from one another.</p> Converting into immovable types</h2> Alright, so we have an idea of how we might be able to construct immovable types in-place, provided as a constructor defined on a type. Now while that's nice, we've also lost an important property with that: whenever GivePatsFuture</code> is constructed, it needs to have a fixed place in memory. Where before we could freely move it around until we started .await</code>ing it.</p> One of the main reasons why async is useful is because it enables ad-hoc concurrent execution</a>. That means we want to be able to take futures and pass them to concurrency operations to enable concurrency through composition. We can't move futures which have a fixed location in memory, so we need a brief moment where futures can be moved before they're ready to be kept in place and polled to completion.</p> The way Pin</code> works with this today is that a type can be !Unpin</code> - but that only becomes relevant once it's placed inside of a Pin</code> structure. With futures that typically doesn't happen until it begins being polled, usually via .await</code>, and so we get the liberty of moving !Unpin</code> futures around until we start .await</code>ing them. That's why !Unpin</code> doesn't mark: "A type which cannot be moved", it marks: "A type which cannot be moved once it has been pinned". This is definitely confusing, so don't worry if it's hard to follow.</p> fn </span>foo</span><T: Unpin>(</span>t</span>: &</span>mut</span> T); </span>// Type is not pinned, type can be moved. </span>fn </span>foo</span><T: </span>!</span>Unpin>(t: &</span>mut</span> T); </span>// Type is not pinned, type can be moved. </span>fn </span>foo</span><T: Unpin>(</span>t</span>: Pin<&</span>mut</span> T>); </span>// Type is pinned, type can be moved. </span>fn </span>foo</span><T: </span>!</span>Unpin>(t: Pin<&</span>mut</span> T>); </span>// Type is pinned, type can't be moved. </span></code></pre> If we want "immovability" to be unconditionally part of a type, we can't make it behave the same way Unpin</code> does. Instead it seems better to separate the movable / immovable requirements into two separate types. We first construct a type which can be freely moved around - and once we're ready to drive it to completion, we convert it to a type which is immovable and we begin calling that. This maps perfectly to the separation between IntoFuture</code> and Future</code> we already use.</p> Let's take a look at our first example again, but modify it slightly. What I'm proposing here is that rather than give_pats</code> returning an impl Future</code>, it should instead return an impl IntoFuture</code>. This type is not pinned, and can be freely moved around. It's only once we're ready to .await</code> it that we call .into_future</code> to obtain the immovable future - and then we call that.</p> struct </span>GivePatsFuture { ... } </span>impl </span>GivePatsFuture { </span> </span>fn </span>new</span>() -> super Pin<&</span>'super mut Self</span>> { ... } </span>// suspend belief pls </span>} </span> </span>struct </span>GivePatsIntoFuture; </span>impl </span>IntoFuture </span>for </span>GivePatsIntoFuture { </span> </span>type </span>Output = (); </span> </span>type </span>IntoFuture = GivePatsFuture; </span> </span> </span>// We call the `Future::new` constructor which gives us a </span> </span>// `Pin<&'super GivePatsFuture>`, and then rather than writing </span> </span>// it into the current function's stack frame we write it in the </span> </span>// caller's stack frame. </span> </span>// </span> </span>// (keep belief suspended a little longer) </span> </span>fn </span>into_future</span>(</span>self</span>) -> super Pin<&</span>'super mut</span> GivePatsFuture> { </span> GivePatsFuture::new() </span>// create in caller's scope </span> } </span>} </span></code></pre> Just like we can keep returning values from functions to pass them further up the call stack, so should we be able to use out-pointers / emplacement / spicy allocate in a stack frame further up the call stack. Though even if we didn't support that out of the gate, we could probably in-line the GivePatsFuture::new</code> into GivePatsIntoFuture::into_future</code> and things would still work. And with that, our .await</code> desugaring could then look something like this:</p> async </span>fn </span>main</span>() { </span> </span>let</span> into_future: GivePatsIntoFuture = </span>give_pats</span>(); </span> </span>let mut</span> future: Pin<&</span>mut</span> GivePats> = GivePatsIntoFuture.</span>into_future</span>(); </span> </span>loop </span>{ </span> </span>match</span> future.</span>poll</span>(&</span>mut</span> current_context) { </span> Poll::Ready(ready) => </span>break</span> ready, </span> Poll::Pending => </span>yield</span> Poll::Pending, </span> } </span> } </span>} </span></code></pre> To re-iterate why this section exists: we can get the same functionality Pin</code> + Unpin</code> provide today by creating two separate types. One type which can be freely moved around. And another type which once constructed will not move locations in memory.</p> So far the only framing of "immovable types" I've seen so far is a single types which have both these properties - just like Unpin</code> does today. What I'm trying to articulate here is that we can avoid that issue if we choose to create two types instead, enabling one to construct the other, and make them provide separate guarantees. I think that's a novel insight, and one I thought was important to spend some time on.</p> Immovable types</h2> Alright, I've been asking folks to suspend belief that we can in fact perform in-place construction of a Pin<&mut Self></code> type somehow and that would all work out the way we want it to. I'm not sure myself, but for the sake of the narrative of this post it was easier if we just pretended we could for a second.</p> The real solution here, of course, to get rid of Pin</code> entirely. Instead types themselves should be able to communicate whether they have a stable memory location or not. The simplest formulation for this would be to add a new built-in auto-trait, Move</code>, which tells the compiler whether a type can be moved or not.</p> auto </span>trait </span>Move {} </span></code></pre> This is of course not a new idea: we've known about the possibility for Move</code> since at least 2017. That's before I started working on Rust. There were some staunch advocates for that in the Rust community in favor of Move</code>, but ultimately that wasn't the design we ended up going with</a>. I think in hindsight most of will acknowledge that the downsides of Pin</code> are real enough that revisiting Move</code> and working through its limitations seems like a good idea 1</a></sup>. To explain what the Move</code> trait is: it would be a language-level trait which governs access to the following capabilities:</p> ^{1</sup> For anyone looking to assign blame here or pull skeletons out of closets: please don't. The higher-order bit here is that we have Pin</code> today, it clearly doesn't work as well as was hoped at the time, and we'd like to replace it with something better. I think the most interesting thing to explore here is how we can move forward and do better.</p> </div>} The ability to be passed by-value into functions and types.</li> The ability to be passed by mutable reference to mem::swap</code></a>, mem::take</code></a>, and mem::replace</code></a>.</li> The ability to be used with any syntactic equivalents to the earlier points, such as assigning to mutable references, closure captures, and so on.</li> </ul> Conversely, when a type implements !Move</code> they would not have access to any of these capabilities - making it so they cannot be moved once they have a fixed memory location. And by default we would assume in all bounds that types are Move</code>, except for places that explicitly opt-out by using + ?Move</code>. Here are examples of things that constitute moves:</p> // # examples of moving </span> </span>// ## swapping two values </span>let mut</span> x = </span>new_thing</span>(); </span>let mut</span> y = </span>new_thing</span>(); </span>swap</span>(&</span>mut</span> x, &</span>mut</span> y); </span> </span>// ## passing by value </span>fn </span>foo</span><T>(</span>x</span>: T) {} </span>let</span> x = </span>new_thing</span>(); </span>foo</span>(x); </span> </span>// ## returning a value </span>fn </span>make_value</span>() -> Foo { </span> Foo { </span> x: </span>42 </span> } </span>} </span> </span>// ## `move` closure captures </span>let</span> x = </span>new_thing</span>(); </span>thread::spawn(</span>move </span>\|\| { </span> </span>let</span> x = x; </span>}) </span></code></pre> And here are some things that do not constitute moves:</p> // # things that are not moves </span> </span>// ## passing a reference </span>fn </span>take_ref</span><T>(</span>x</span>: &T) {} </span>let</span> x = </span>new_thing</span>(); </span>take_ref</span>(&x); </span> </span>// ## passing mutable references is also okay, </span>// but you have to be careful how you use it </span>fn </span>take_mut_ref</span><T>(</span>x</span>: &</span>mut</span> T) {} </span>let mut</span> x = </span>new_thing</span>(); </span>take_mut_ref</span>(&</span>mut</span> x); </span></code></pre> Passing types by-value will never</em> be compatible with !Move</code> types because that’s what a move is. Passing types by-reference will always</em> be compatible with !Move</code> types because they are immutable 2</a></sup>. The only place with some ambiguity is when we work with mutable references, as things like mem::swap</code> allow us to violate the immovability guarantees.</p> ^{2</sup> Yes yes, we'll get to internal mutability in a second here.</p> </div> If a function wants to take a mutable reference which may be immovable, they will have to add + ?Move</code> to it. If a function does not use + ?Move</code> on their mutable reference, then a !Move</code> type cannot be passed to it. In practice this will work as follows:</p>}fn </span>meow</span><T>(</span>cat</span>: T); </span>// by-value, can't pass `!Move` values </span>fn </span>meow</span><T>(</span>cat</span>: &T); </span>// by-ref, can pass `!Move` values </span>fn </span>meow</span><T>(</span>cat</span>: &</span>mut</span> T); </span>// by-mut-ref, can't pass `!Move` values </span>fn </span>meow</span><T: </span>?</span>Move>(cat: &</span>mut</span> T) </span>// by-mut-ref, can pass `!Move` values </span></code></pre> By default all cat: &mut T</code> bounds would imply + Move</code>. And only where we opt-in to + ?Move</code> could !Move</code> types be passed. In practice it seems likely most places will probably be fine adding + ?Move</code>, since it's far more common to write to a field of a mutable reference than it is to replace it whole-sale using mem::swap</code>. Things like interior mutability are probably also largely fine under these rules, since even if accesses go through shared references, updating the values in the pointers will have to interact with the earlier rules we've set out - and those are safe by default.</p> To be entirely accurate we also have to consider internal mutability. That allows us to mutate values through shared references - but only by being able to conditionally convert it to &mut</code> references at runtime. Just because we allow casting &T</code> to &mut T</code> at runtime, doesn't mean that the rules we've applied to the system don't still work. Say we held an &mut T: !Move</code> inside of a Mutex</a>. If we tried to call the deref_mut</code> method</a>, we'd get a compile-error because that bound hasn't yet declared that T: ?Move</code>. We could probably add that, but because it doesn't work by default we'd have an opportunity to validate its soundness before adding it.</p> Anyway, that's enough theory about how this should probably work for now. Let's try and update our earlier example, replacing Pin</code> with !Move</code>. That should be as simple as adding a !Move</code> impl on GivePatsFuture</code>.</p> struct </span>GivePatsFuture { </span> </span>resume_from</span>: GivePatsState, </span> </span>data</span>: Option<String>, </span> </span>name</span>: Option<&</span>'self str</span>>, </span>} </span>impl </span>!Move for GivePatsFuture {} </span></code></pre> And once we have that, we can change our constructors to return super Self</code> instead of super Pin<&'super mut Self></code>. We already know that emplacement using something like super Self</code> (not actual notation) to write to fixed memory locations seems plausible. All we then need to do is add an auto-trait which tells the type-system that further move operations aren't allowed.</p> struct </span>GivePatsFuture { ... } </span>impl </span>!Move for GivePatsFuture {} </span>impl </span>GivePatsFuture { </span> </span>fn </span>new</span>() -> super </span>Self </span>{ ... } </span>// create in caller's scope </span>} </span> </span>struct </span>GivePatsIntoFuture; </span>impl </span>IntoFuture </span>for </span>GivePatsIntoFuture { </span> </span>type </span>Output = (); </span> </span>type </span>IntoFuture = GivePatsFuture; </span> </span>fn </span>into_future</span>(</span>self</span>) -> super GivePatsFuture { </span> GivePatsFuture::new() </span>// create in caller's scope </span> } </span>} </span></code></pre> I probably should have said this sooner, but I'll say it now: in this post I'm intentionally not bothering with backwards-compat. The point, again, is to break the complicated design space of "immovable types" into smaller problems we can tackle one-by-one. Figuring out how to bridge Pin</code> and !Move</code> is something we will want to figure out at some point - but not now.</p> As far as async {}</code> and Future</code> are concerned: this should work! This allows us to freely move around async blocks which desugar into IntoFuture</code>. And only once we're ready to start polling them do we call into_future</code> to obtain an impl Future + !Move</code>. A system like that is equivalent to the existing Pin</code> system, but does not need Pin</code> in its signature. For good measure, here's how we would be able to rewrite the signature of Future</code> with this change:</p> // The current `Future` trait </span>// using `Pin<&mut Self>` </span>pub trait </span>Future { </span> </span>type </span>Output; </span> </span>fn </span>poll</span>(</span>self</span>: Pin<&</span>mut Self</span>>, </span>cx</span>: &</span>mut </span>Context<'_>) -> Poll<</span>Self::</span>Output>; </span>} </span> </span>// The `Future` trait leveraging `Move` </span>// using `&mut self` </span>pub trait </span>Future { </span> </span>type </span>Output; </span> </span>fn </span>poll</span>(&</span>mut </span>self</span>, </span>cx</span>: &</span>mut </span>Context<'_>) -> Poll<</span>Self::</span>Output>; </span>} </span></code></pre> That would also mean: no more Pin</code>-projections. No more incompatibilities with Drop</code>. Because it's an auto-trait that governs language behavior, as long as the base rules are sound, the interaction with all other parts of Rust would be sound.</p> Also, most relevant for me probably, this would make it possible to write future state machines using methods</em> and functions</em>, rather than the current status quo where we just lump everything into the poll</code> function body. After having written a ridiculous amount of futures by hand over the past six years, I can't tell you how much I'd love to be able to do that.</p> Motivating example, reworked</h2> Now that we've covered self-referential lifetimes, in-place construction, understand async {}</code> should return IntoFuture</code>, and have seen !Move</code>, we're ready to bring these features together to rework our motivating example. This is what we started with using regular async/.await</code> code:</p> async </span>fn </span>give_pats</span>() { </span> </span>let</span> data = "</span>chashu tuna</span>".</span>to_string</span>(); </span> </span>let</span> name = data.</span>split</span>(' ').</span>next</span>().</span>unwrap</span>(); </span> </span>pat_cat</span>(&name).await; </span> println!("</span>patted </span>{name}</span>"); </span>} </span> </span>async </span>fn </span>main</span>() { </span> </span>give_pats</span>().await; </span>} </span></code></pre> And here is what that, with these new capabilities, here is what async fn give_pats</code> would be able to desugar to. Note the 'self</code> lifetime, the !Move</code> impl for the future, the omission of Pin</code> everywhere, and the in-place construction of the type.</p> enum </span>GivePatsState { </span> Created, </span> Suspend1, </span> Complete, </span>} </span> </span>struct </span>GivePatsFuture { </span> </span>resume_from</span>: GivePatsState, </span> </span>data</span>: Option<String>, </span> </span>name</span>: Option<&</span>'self str</span>>, </span>// ← Note the `'self` lifetime </span>} </span>impl </span>!Move for GivePatsFuture {} </span>// ← This type is immovable </span>impl </span>Future </span>for </span>GivePatsFuture { </span> </span>type </span>Output = (); </span> </span>fn </span>poll</span>(&</span>mut </span>self</span>, </span>cx</span>: &</span>mut </span>Context<'_>) </span>// ← No `Pin` needed </span> -> Poll<</span>Self::</span>Output> { ... } </span>} </span> </span>struct </span>IntoGivePatsFuture {} </span>impl </span>IntoFuture </span>for </span>IntoGivePatsFuture { </span> </span>type </span>Output = (); </span> </span>type </span>IntoFuture: GivePatsFuture </span> </span>fn </span>into_future</span>(</span>self</span>) </span> -> super GivePatsFuture { </span>// ← Writes to a stable addr </span> </span>Self </span>{ </span> resume_from: GivePatsState::Created, </span> data: None, </span> name: None, </span> } </span> } </span>} </span></code></pre> And finally, we can then desugar the give_pats().await</code> call to concrete types we construct and call to completion:</p> let</span> into_future = IntoGivePatsFuture {}; </span>let mut</span> future = into_future.</span>into_future</span>(); </span>// ← Immovable without `Pin` </span>loop </span>{ </span> </span>match</span> future.</span>poll</span>(&</span>mut</span> current_context) { </span> Poll::Ready(ready) => </span>break</span> ready, </span> Poll::Pending => </span>yield</span> Poll::Pending, </span> } </span>} </span></code></pre> And with that, we should have a working example of async {}</code> blocks, desugared to concrete types and traits that don't use Pin</code> anywhere at all. Accessing fields within it wouldn't go through any kind of pin projection, and there would no longer be any need for things like stack-pinning. Immovability would just be a property of the types themselves, constructed when we need them in the place where we want to use them.</p> Oh and I guess just to mention it: functions working with these traits would always want to use T: IntoFuture</code> rather than T: Future</code>. That's not a big change, and actually something people should already be doing today. But I figured I'd mention it in case people are confused about what the bounds should be for concurrency operations.</p> Phased initialization</h2> We didn't show this in our example, but there is one more aspect to self-referential types worth covering: phased initialization. This is when you initialize parts of a type at separate points in time. In our motivating example we didn't have to use that, because the self-references lived inside of an Option</code>. That means that when we initialized the type we could just pass None</code>, and things were fine. However, say we did want to initialize a self-reference, how would we go about that?</p> struct </span>Cat { </span> </span>data</span>: String, </span> </span>name</span>: &</span>'self str</span>, </span>} </span>impl </span>!Move for Cat {} </span> </span>impl </span>Cat { </span> </span>fn </span>new</span>(</span>data</span>: String) -> super </span>Self </span>{ </span> Cat { </span> data: "</span>chashu tuna</span>".</span>to_string</span>(), </span> name: </span>/ How do we reference `self.data` here? / </span> } </span> } </span>} </span></code></pre> Now of course because String</code> is heap-allocated, its address is actually stable and so we could write something like this:</p> struct </span>Cat { </span> </span>data</span>: String, </span> </span>name</span>: &</span>'self str</span>, </span>} </span>impl </span>!Move for Cat {} </span> </span>impl </span>Cat { </span> </span>fn </span>new</span>(</span>data</span>: String) -> super </span>Self </span>{ </span> </span>let</span> data = "</span>chashu tuna</span>".</span>to_string</span>(); </span> Cat { </span> name: data.</span>split</span>(' ').</span>next</span>().</span>unwrap</span>(), </span> data, </span> } </span> } </span>} </span></code></pre> That's clearly cheating, and not what we want people to have to do. But it does point us at how the solution here should probably work: we first need a stable address to point to. And once we have that address, we can refer to it. We can't do that if we have to build the entire thing in a single go. But what if we could do it in multiple phases</em>? That's what Niko's recent post on borrow checking and view types</a> went into. That would allow us to change our example to instead be written like this:</p> struct </span>Cat { </span> </span>data</span>: String, </span> </span>name</span>: &</span>'self str</span>, </span>} </span>impl </span>!Move for Cat {} </span> </span>impl </span>Cat { </span> </span>fn </span>new</span>(</span>data</span>: String) -> super </span>Self </span>{ </span> </span>super let</span> this = Cat { data: "</span>chashu tuna</span>".</span>to_string</span>() }; </span>// ← partial init </span> this.name = this.data.</span>split</span>(' ').</span>next</span>().</span>unwrap</span>(); </span>// ← finish init </span> this </span> } </span>} </span></code></pre> We initialize the owned data in Cat</code> first. And once we have that, we can then initialize the references to it. These references would be 'self</code>, we sprinkle in a super let</code> annotation to indicate we're placing this in the caller's scope, and everything should subsequently check out.</p> Migrating from Pin to Move</h2> What we didn't cover in this post is any migration story from the existing Pin</code>-based APIs to the new Move</code>-based system. If we want to move off of Pin</code> in favor of Move</code>, the only path plausible way I see is by minting new traits that don't carry Pin</code> in its signature, and providing bridging impls from the old traits to the new traits. A basic conversion with an explicit method could look like this, though blanket impls could also be a possibility:</p> pub trait </span>NewFuture { </span> </span>type </span>Output; </span> </span>fn </span>poll</span>(&</span>mut </span>self</span>, ...) { ... } </span>} </span> </span>pub trait </span>Future { </span> </span>type </span>Output; </span> </span>fn </span>poll</span>(</span>self</span>: Pin<&</span>mut Self</span>>, ...) { ... } </span> </span> </span>/// Convert this future into a `NewFuture`. </span> </span>fn </span>into_new_future</span>(</span>self</span>: Pin<&</span>mut Self</span>>) -> NewFutureWrapper<&</span>mut Self</span>> { ... } </span>} </span> </span>/// A wrapper bridging the old future trait to the new future trait. </span>struct </span>NewFutureWrapper<</span>'a</span>, F: Future>(Pin<&</span>'a mut</span> F>); </span>impl </span>!Move for NewFutureWrapper {} </span>impl</span><</span>'a</span>, F> NewFuture </span>for </span>NewFutureWrapper<</span>'a</span>, F> { ... } </span></code></pre> I've been repeating this line for at least three years now: but if we want to fix the problems with Pin</code>, the first step we need to take is to not make the problem worse. If the stdlib needs to fix the Future</code> trait once, that sucks but it's fine and we'll find a way to do it. But if we tie Pin</code> up into a number of other traits, the problems will compound and I'm no longer sure whether we can rid ourselves of Pin</code>. And that's a problem, because Pin</code> is broadly disliked and we actively want to get rid of it.</p> Compatibility with self-referential types is not only relevant for iteration; it's a generalized property which ends up interacting with nearly every trait, function, and language feature. Move</code> just composes with any other trait, and so there's no need for a special PinnedRead</code> or anything. A type would instead just implement Read + Move</code>, and that would be enough for a self-referential reader to function. And we can repeat that for any other combination of traits.</p> In-place construction of course does change the signature of traits. But in order to support that in a backwards-compatible way, all we'd need to do is enable traits to opt-in to "can perform in-place construction"</em>. And being able to gradually roll-out capabilities like that is exactly why we're working on effect generics.</p> pub trait </span>IntoFuture { </span> </span>type </span>Output; </span> </span>type </span>IntoFuture: Future<Output = </span>Self::</span>Output>; </span> </span> </span>// Marked as compatible with in-place construction, with </span> </span>// implementations being able to decide whether they want </span> </span>// to use it or not. </span> </span>fn </span>into_future</span>(</span>self</span>) -> #[</span>maybe</span>(super)] </span>Self</span>::IntoFuture; </span>} </span></code></pre> If we want self-referential types to be generally useful, they need to practically compose with most other features we have. And so really, the first step to getting there is stop stabilizing any new traits in the stdlib which use Pin</code> in its signature.</p> Making immovable types movable</h2> So far we’ve been talking a lot about self-referential types and how we need to make sure they cannot be moved, because moving them would be bad. But what if we did allow them to be moved? In C++ this is possible using a feature called "move constructors"</em>, and if we supported self-referential types in Rust, it doesn't seem like a big leap to support that too.</p> Before we go any further I want to preface this: I've heard from people who have worked with move constructors in C++ that they can be rather tricky to work with. I haven't worked with them, so I can't speak from experience. Personally I don't really have any uses where I feel like I would have wanted move constructors, so I'm not particularly in favor or against supporting them. I'm writing this section mostly out of academic interest, because I know there will be people wondering about this. And the rules for how this should work seem fairly straightforward.</p> Niko Matsakis recently wrote a two-parter on the Claim</code> trait (first</a>, second</a>), proposing a new Claim</code> trait to fill the gap between Clone</code> and Copy</code>. This trait would be for types which are "cheap" to clone, such as the Arc</code> and Rc</code> types. And using autoclaim</a>, the compiler would automatically insert calls to .claim</code> as needed. For example when a move \|\|</code> closure captures a type implementing Claim</code> but it is already in use somewhere else - it would automatically call .claim</code> so it would compile.</p> Enabling immovable types to relocate would work much the same as auto-claiming would. We would need to introduce a new trait, which we'll call Relocate</code> here, with a method relocate</code>. Whenever we tried to move an otherwise immovable value, we would automatically call .relocate</code> instead. The signature of the Relocate</code> trait would take self</code> as a mutable reference. And return an instance of Self</code>, constructed in-place:</p> trait </span>Relocate { </span> </span>fn </span>relocate</span>(&</span>mut </span>self</span>) -> super </span>Self</span>; </span>} </span></code></pre> Note the signature of self</code> here: we take it by mutable reference - not owned nor shared. That is because what we're writing is effectively the immovable equivalent to Into</code>, but we can't take self by-value - so we have to take it by-reference instead and tell people to just mem::swap</code> away. Applying this our earlier Cat</code> example, we would be able to implement it as follows:</p> struct </span>Cat { </span> </span>data</span>: String, </span> </span>name</span>: &</span>'self str</span>, </span>} </span>impl </span>Cat { </span> </span>fn </span>new</span>(</span>data</span>: String) -> super </span>Self </span>{ ... } </span>} </span>impl </span>Relocate </span>for </span>Cat { </span> </span>fn </span>relocate</span>(&</span>mut </span>self</span>) -> super </span>Self </span>{ </span> </span>let mut</span> data = String::new(); </span>// dummy type, does not allocate </span> mem::swap(&</span>mut </span>self</span>.data, &</span>mut</span> data); </span>// take owned data </span> </span>super let</span> cat = Cat { data }; </span>// construct new instance </span> cat.name = cat.data.</span>split</span>(' ').</span>next</span>().</span>unwrap</span>(); </span>// create self-ref </span> cat </span> } </span>} </span></code></pre> We're making one sketchy assumption here: we need to be able to take the owned data from self</code>, without running into the issue where the data can't be moved because it is already borrowed from self</code>. This is a general problem we need to solve, and one way we could for example work around this is by creating dummy pointers in the main struct to ensure the types are always valid - but we invalidate the types:</p> struct </span>Cat { </span> </span>data</span>: String, </span> </span>dummy_data</span>: String, </span>// never initialized with a value </span> </span>name</span>: &</span>'self str</span>, </span>} </span> </span>impl </span>Relocate </span>for </span>Cat { </span> </span>fn </span>relocate</span>(&</span>mut </span>self</span>) -> super </span>Self </span>{ </span> </span>self</span>.name = &</span>self</span>.dummy_data; </span>// no more references to `self.data` </span> </span>let</span> data = mem::take(&</span>mut </span>self</span>.data); </span>// shorter than `mem::swap` </span> </span>super let</span> cat = Cat { data }; </span> cat.name = cat.data.</span>split</span>(' ').</span>next</span>().</span>unwrap</span>(); </span> cat </span> } </span>} </span></code></pre> In this example the Cat</code> would implement Move</code> even if it has a 'self</code> lifetime, because we can freely move around. When a type is dropped after having been passed to Relocate</code>, it should not call its Drop</code> impl. Because semantically we're not trying to drop the type - all we're doing is updating its location in memory. Under these rules access to 'self</code> in structs would be available both if Self: !Move</code> OR Self: Relocate</code>.</p> I want to again emphasize that I'm not directly advocating here for the introduction of move constructors to Rust. Personally I'm pretty neutral about them, and I can be convinced either way. I mainly wanted to have at least once walked through the out the way move constructors could work, because it seems like a good idea to know a gradual path here should be possible. Hopefully that point is coming across okay here.</p> Further reading</h2> The Pin</code> RFC</a> is an interesting read as it describes the system for immovable types we ended up going with today. Specifically the comparison</a> between Pin</code> and Move</code>, and section on drawbacks</a> are interesting to read back up on. Especially when we compare it with the pin docs</a>, and see what was not present in the RFC - but later turned out to be major practical issues (e.g. pin projections, interactions with the rest of the language).</p> Tmandry presented an interesting series (blog 1</a>, blog 2</a>, talk</a>) on async internals. Specifically he covers how async {}</code> blocks desugar into Future</code>-based state machines. This post uses that desugaring as its motivating example, so for those keen to learn more about what happens behind the scenes, this is an excellent resource. The section on .await</code> desugaring</a> in the Rust reference is also a good read, as it captures the status quo in the compiler.</p> More recently Miguel Young de la Sota's (mcyoung) talk</a> and crate</a> to support C++ move constructors is interesting to read up on. Something I haven't fully processed yet, but is interesting, is the New</code> trait</a> it exposes. This can be used to construct types in-place on both the stack and the heap</a>, which ideally something like the super let</code>/super Type</code> notation could support too. You can think of C++'s move constructors as further evolution of immovable types, so it's no surprise that there are lots of shared concepts.</p> Two years ago I tried formulating a way we could leverage view types for safe pin projection (post</a>), and I came up short in a few regards. In particular I wasn't sure how to deal with the interactions with #[repr(packed)]</code>, how to make Drop</code> compatible, and how to mark Unpin</code> as unsafe</code>. There might be a path for the latter, but I'm not aware of any practical solutions for the first two issues. This post is basically a sequel to that post, but changing the premise from: "How can we fix Pin</code>?" to "How can we replace Pin</code>?".</p> Niko's series on view types is also well worth reading. His first post</a> discusses what view types are, how they'd work, and why they're useful. And in one of his most recent posts</a> he discusses how view types fall into a broader "4-part roadmap for the borrow checker" (aka: "the borrow checker within"). In his last post he directly covers phased initialization using view types as well, which is one of the features we discuss in this post in relation to self-referential types.</p> Finally I'd suggest looking at the ouroboros crate</a>. It enables safe, phased initialization for self-referential types on stable Rust by leveraging macros and closures. The way it works is that fields using owned data are initialized first. And then the closures are executed to initialize fields referencing the data. Phased initialization using view types as described in this post emulates that approach, but enables it directly from the language through a generally useful feature.</p> Conclusion</h2> In this post we've deconstructed "self-referential types" into four constituent parts:</p> 'unsafe</code> (unchecked) and 'self</code> (checked) lifetimes which will make it possible to express self-referential lifetimes.</li> A moral equivalent to super let</code> / -> super Type</code> to safely support out-pointers.</li> A way to backwards-compatibly add optional -> super Type</code> notations.</li> A new Move</code> auto-trait which governs access to move operations.</li> A view types feature which will make it possible to construct self-referential types without going through an Option</code> dance.</li> </ol> The final insight this post provides is that today's Pin</code> + Unpin</code> system can be emulated with Move</code> by creating Move</code> wrappers which can return !Move</code> types. In the context of async, the pattern would be to construct an impl IntoFuture + Move</code> wrapper, which constructs an impl Future + !Move</code> future in-place via an out-pointer.</p> People generally dislike Pin</code>, and as far as I can tell there is broad support for exploring alternative solutions such as Move</code>. Right now the only trait that uses Pin</code> in the stdlib is Future</code>. In order to facilitate a migration off of Pin</code> to something like Move</code>, we would do well not to further introduce any Pin</code>-based APIs to the stdlib. Migrating off of a single API will take effort, but seems ultimately doable. Migrating off of a host of APIs will take more effort, and makes it more likely we'll forever be plagued with the difficulties of Pin</code>.</p> The purpose of this post has been to untangle the big scary problem of "immovable types" into its constituents parts so we can begin tackling them one-by-one. None of the syntax or semantics in this post are meant to be concrete or final. I mainly wanted to have at least once walked through everything required to make immovable types work - so that others can dig in, think along, and we can begin refining concrete details.</p> in-place construction seems surprisingly simple? Sat, 22 Jun 2024 00:00:00 +0000 introduction</h2> I've been thinking a little bit about self-referential types recently, and one of its requirements is that a type, when constructed, has a fixed location in memory 1</a></sup>. That's needed, because a reference is pointer to a memory address - and if the memory address it points to changes, that would invalidate the pointer, which can lead to undefined behavior.</p> ^{1</sup> Eric Holk and I toyed around for a little while with the idea of offset-schemes, where we only track offsets from the start of structs. But that gets weird once you involve non-linear memory such as when using references in the heap. Using real, actual addresses is probably the better solution since that should work in all cases - even if it brings its own sets of challenges.</p> </div>}Where this becomes tricky is that returning data from a function constitutes a move</em>. A constructor which creates a type within itself and returns it, will change the address of the type it constructs. Take this program which constructs a type Cat</code> and in the function Cat::new</code> (playground</a>):</p> use </span>std::ptr::addr_of; </span> </span>struct </span>Cat { </span>age</span>: </span>u8 </span>} </span>impl </span>Cat { </span> </span>fn </span>new</span>(</span>age</span>: </span>u8</span>) -> </span>Self </span>{ </span> </span>let</span> this = </span>Self </span>{ age }; </span> dbg!(addr_of!(this)); </span>// ← first call to `addr_of!` </span> this </span> } </span>} </span> </span>fn </span>main</span>() { </span> </span>let</span> cat = Cat::new(</span>4</span>); </span> dbg!(addr_of!(cat)); </span>// ← second call to `addr_of!` </span>} </span></code></pre> If we run this program we get the following output:</p> [src/main.rs:7:9] addr_of!(this) = 0x00007ffe0b3575d7 # first call </span>[src/main.rs:14:5] addr_of!(cat) = 0x00007ffe0b357747 # second call </span></code></pre> This shows that the address of Cat</code> within</em> Cat::new</code> is different from the address once it has been returned from the function. Returning types from functions means changing addresses. Languages like C++ have ways to work around this by providing something called move constructors</em>, enable types to update their own internal addresses when they are moved in memory. But instead, what if we could just construct types in-place so they didn't have to move in the first place?</p> in-place construction</h2> We already perform in-place construction when desugaring async {}</code> blocks, so this is something we know how to do. And in the ecosystem there is also the moveit</a> and ouroboros</a> crates. These are all great, but they all do additional things like "self-references" or "move constructors". Constructing in-place rather than returning from a function can be useful just for the sake of reducing copies - so let's roll our own version of just that.</p> The way we can do this is by creating a stable location in memory we can store our value in. Rather than returning a value, a constructor should take a mutable reference to this memory location and write directly into it instead. And because we're starting with a location</em> and write into it later, this location needs to be MaybeUninit</code>. If we put those pieces together, we can adapt our earlier example to the following (playground</a>):</p> use </span>std::ptr::{addr_of, addr_of_mut}; </span>use </span>std::mem::MaybeUninit; </span> </span>struct </span>Cat { </span>age</span>: </span>u8 </span>} </span>impl </span>Cat { </span> </span>fn </span>new</span>(</span>age</span>: </span>u8</span>, </span>slot</span>: &</span>mut </span>MaybeUninit<</span>Self</span>>) { </span> </span>let</span> this: </span>mut Self </span>= slot.</span>as_mut_ptr</span>(); </span> </span>unsafe </span>{ </span> addr_of_mut!((this).age).</span>write</span>(age); </span> dbg!(addr_of!(this)); </span>// ← second call to `addr_of!` </span> }; </span> } </span>} </span> </span>fn </span>main</span>() { </span> </span>let mut</span> slot = MaybeUninit::uninit(); </span> dbg!(addr_of!(slot)); </span>// ← first call to `addr_of!` </span> Cat::new(</span>4</span>, &</span>mut</span> slot); </span> </span>let</span> cat: &</span>mut</span> Cat = </span>unsafe </span>{ (slot).</span>assume_init_mut</span>() }; </span> dbg!(addr_of!(cat)); </span>// ← third call to `addr_of!` </span>} </span></code></pre> If we run the program it will print the following:</p> [src/main.rs:15:5] addr_of!(slot) = 0x00007ffc9daa590f # first call </span>[src/main.rs:9:9] addr_of!(this) = 0x00007ffc9daa590f # second call </span>[src/main.rs:18:5] addr_of!(cat) = 0x00007ffc9daa590f # third call </span></code></pre> To folks who aren't used to writing unsafe Rust this might look a little overwhelming. But what we've done is a fairly mechanical translation. Rather than returning the type Self</code>, we're created a MaybeUninit<Self></code> and passed it by-reference. The constructor then writes into it, initializing the memory. From that point onward, all references to Cat</code> are valid and can be assumed to be initialized.</p> Unfortunately calling assume_init</code> on the actual value</em> of Cat</code> is not possible because the compiler treats that as a move - which makes sense since it takes a type and returns another. But that's mostly a limitation of how we're doing things - not what we're doing.</p> indirect in-place construction</h2> Now what happens if there is a degree of indirection? What if rather than construct just a Cat</code>, we want to construct a Cat</code> inside of a Bed</code>. We would have to take the memory location of the outer type, and use that as the location for the inner type. Let's extend our first example by doing exactly that (playground</a>):</p> use </span>std::ptr::addr_of; </span> </span>struct </span>Bed { </span>cat</span>: Cat } </span>impl </span>Bed { </span> </span>fn </span>new</span>() -> </span>Self </span>{ </span> </span>let</span> cat = Cat::new(</span>4</span>); </span> </span>Self </span>{ cat } </span> } </span>} </span> </span>struct </span>Cat { </span>age</span>: </span>u8 </span>} </span>impl </span>Cat { </span> </span>fn </span>new</span>(</span>age</span>: </span>u8</span>) -> </span>Self </span>{ </span> </span>let</span> this = </span>Self </span>{ age }; </span> dbg!(addr_of!(this)); </span>// ← first call to `addr_of!` </span> this </span> } </span>} </span> </span>fn </span>main</span>() { </span> </span>let</span> bed = Bed::new(); </span> dbg!(addr_of!(bed)); </span>// ← second call to `addr_of!` </span> dbg!(addr_of!(bed.cat)); </span>// ← third call to `addr_of!` </span>} </span></code></pre> If we run the program it will print the following:</p> [src/main.rs:15:9] addr_of!(this) = 0x00007fff3910702f </span>[src/main.rs:22:5] addr_of!(bed) = 0x00007fff391071b7 </span>[src/main.rs:23:5] addr_of!(bed.cat) = 0x00007fff391071b7 </span></code></pre> Adapting our return-based example to preserve referential stability is once again very mechanical. Rather than returning Self</code> from a function, we pass a mutable reference to MaybeUninit<Self></code>. In for Cat</code> to be constructed in Bed</code>, all we have to do is make sure Bed</code> contains a slot for Cat</code> to be written to. Put together we end up with the following (playground</a>):</p> use </span>std::ptr::{addr_of, addr_of_mut}; </span>use </span>std::mem::MaybeUninit; </span> </span>struct </span>Bed { </span>cat</span>: MaybeUninit<Cat> } </span>impl </span>Bed { </span> </span>fn </span>new</span>(</span>slot</span>: &</span>mut </span>MaybeUninit<</span>Self</span>>) { </span> </span>let</span> this: </span>mut Self </span>= slot.</span>as_mut_ptr</span>(); </span> Cat::new(</span>4</span>, </span>unsafe </span>{ &</span>mut </span>(this).cat }); </span> } </span>} </span> </span>struct </span>Cat { </span>age</span>: </span>u8 </span>} </span>impl </span>Cat { </span> </span>fn </span>new</span>(</span>age</span>: </span>u8</span>, </span>slot</span>: &</span>mut </span>MaybeUninit<</span>Self</span>>) { </span> </span>let</span> this: </span>mut Self </span>= slot.</span>as_mut_ptr</span>(); </span> </span>unsafe </span>{ </span> addr_of_mut!((this).age).</span>write</span>(age); </span> dbg!(addr_of!(this)); </span>// ← second call to `addr_of!` </span> }; </span> } </span>} </span> </span>fn </span>main</span>() { </span> </span>let mut</span> slot = MaybeUninit::uninit(); </span> dbg!(addr_of!(slot)); </span>// ← first call to `addr_of!` </span> Bed::new(&</span>mut</span> slot); </span> </span>let</span> bed: &</span>mut</span> Bed = </span>unsafe </span>{ (slot).</span>assume_init_mut</span>() }; </span> dbg!(addr_of!(bed)); </span>// ← third call to `addr_of!` </span>} </span></code></pre> Which if we run the program will print the following addresses. These are all the same because Cat</code> is the only field inside of Bed</code>, so they happen to point to the same memory location:</p> [src/main.rs:23:5] addr_of!(slot) = 0x00007fff8271d86f # first call </span>[src/main.rs:17:9] addr_of!(this) = 0x00007fff8271d86f # second call </span>[src/main.rs:26:5] addr_of!(bed) = 0x00007fff8271d86f # third call </span></code></pre> future possibilities</h2> If we squint here it's not hard to see how this could be converted into a language feature. In this post we've mechanically performed a transformation by hand. Rather than returning a type T</code> from a function, we're taking a &mut MaybeUninit<T></code> and writing into that. It feels like it's basically just a spicy return</em>; and it seems like something we could introduce some kind of notation for.</p> Though admittedly things get trickier once we want to also enable self-references, perform phased initialization, immovable types, and so on. But those all depend on being able to write to a fixed place in memory - and it feels like perhaps these are concepts which we can decouple from one another? Anyway, if we just take in-place construction as a feature, I think we might be able to get away with something like this for our first example:</p> use </span>std::ptr::addr_of; </span> </span>struct </span>Cat { </span>age</span>: </span>u8 </span>} </span>impl </span>Cat { </span> </span>fn </span>new</span>(</span>age</span>: </span>u8</span>) -> #[</span>in_place</span>] </span>Self </span>{ </span>// ← new notation </span> </span>Self </span>{ age } </span> } </span>} </span> </span>fn </span>main</span>() { </span> </span>let</span> cat = Cat::new(</span>4</span>); </span> </span>// ^ cat was constructed in-place </span>} </span></code></pre> This is obviously not a new idea - but it stood out to me how simple the actual underpinnings of in-place construction seem. The change from a regular return to an in-place return feels mechanical in nature - and that seems like a good sign. For good measure let's also adapt our second example:</p> use </span>std::ptr::addr_of; </span> </span>struct </span>Bed { </span>cat</span>: Cat } </span>impl </span>Bed { </span> </span>fn </span>new</span>() -> #[</span>in_place</span>] </span>Self </span>{ </span>// ← new notation </span> </span>Self </span>{ cat: Cat::new(</span>4</span>) } </span> </span>// ^ cat was constructed in-place </span> } </span>} </span> </span>struct </span>Cat { </span>age</span>: </span>u8 </span>} </span>impl </span>Cat { </span> </span>fn </span>new</span>(</span>age</span>: </span>u8</span>) -> #[</span>in_place</span>] </span>Self </span>{ </span>// ← new notation </span> </span>Self </span>{ age } </span> } </span>} </span> </span>fn </span>main</span>() { </span> </span>let</span> bed = Bed::new(); </span> </span>// ^ bed was constructed in-place </span>} </span></code></pre> Admittedly I haven't read up on the 10-year discourse of placement-new</a>, so I assume there are plenty of details left out that make this hard in the general sense. Things like heap-addresses and intermediate references. But for the simplest case? It seems surprisingly doable. Not quite something which we can proc-macro - but not far off either. And maybe scoping that as a first target would be enough? I don't know.</p> The connection to super let</h2> edit 2024-06-25</strong>: This section was added after first publishing this post.</em></p> Jack Huey reached out after publishing this post, and mentioned there might be a connection with the super let</code> feature</a>. I think that's a super interesting point, and it's not hard to see why! Take this example from Mara's post:</p> let</span> writer = { </span> println!("</span>opening file...</span>"); </span> </span>let</span> filename = "</span>hello.txt</span>"; </span> </span>super let</span> file = File::create(filename).</span>unwrap</span>(); </span> Writer::new(&file) </span>}; </span></code></pre> The super let file</code> notation here allows the file</code>'s lifetime to be scoped to the outer scope, making it valid for Writer</code> to take a reference to file</code> and return. Without super let</code> this would result in a lifetime error. Its vibes are very similar to the #[in_place]</code> notation we posited in this post.</p> Perhaps there a synthesis of both features could exist to create a form of "generalized super scope" feature? There definitely appears like there might be some kind of connection. Like, we could imagine writing something like super let</code> to denote a "value which is allocated in the caller's frame"</em>. And a function returning super Type</code> to signal from the type signature that this is actually an out-pointer 2</a></sup>.</p> ^{2</sup> Shout out to James Munns for teaching me about the term "outpointer" - that's apparently the term C++ uses for this using the outptr</code> keyword.</p> </div>}use </span>std::ptr::addr_of; </span> </span>struct </span>Cat { </span>age</span>: </span>u8 </span>} </span>impl </span>Cat { </span> </span>fn </span>new</span>(</span>age</span>: </span>u8</span>) -> super </span>Self </span>{ </span> </span>super let</span> this = </span>Self </span>{ age }; </span>// declare in the caller's frame </span> dbg!(addr_of!(this)); </span> this </span> } </span>} </span></code></pre> It's worth nothing though that this is not an endorsement for actually going with super let</code> or super Type</code> - but merely to speculate how there might be a possible connection between both features. I think it's fun, and the connection between both seems worthy of further exploration!</p> conclusion</h2> In this post I've shown how we can construct types in-place using MaybeUninit</code> which surprised me how simple it ended up being. I mostly wanted to have gone through the motions at least once - and now I have, and that was fun!</p> edit 2024-06-22:</strong> Thanks to Jordan Rose and Simon Sapin for helping un-break the unsafe pointer code in an earlier version of this post. Goes to show: Rust's pointer ergonomics really could use an overhaul</a>.</p> Context Managers: Undroppable Types for Free Wed, 05 Jun 2024 00:00:00 +0000 In program language design I'm a big fan of features which fall out of other, more general features. People occasionally talk about both "unleakable" and "undroppable" types. I see a lot of value in "unleakable" types because if a type is "unleakable" we can guarantee the type will always have its destructor called. This would allow us to use Drop</code> to be used to uphold safety invariants, which will make it possible to write things like "task scopes". For type system nerds: yes, this means Rust would be able to uphold linear invariants via its type system, AKA equip Rust with linear types.</p> "Undroppable types" have a more limited application. There is some element of undroppability needed to ensure async destructors can only be called in async contexts, but that's not the same things as a generalized system for "undroppable types". The main use for undroppable types is to be able to guarantee</em> that some type, when destructed, always either takes or returns some kind of value.</p> ^{1</sup></div>}I believe it should be possible to express "undroppable types" using Tmandry's design for context/capabilities</a>. This design introduces a system similar to Python's "context manager" API. That systems enables additional arguments to be passed to specific trait impls (like we do with thread-locals today, but via function signatures), which must be in-scope when the trait is invoked. Assuming the implementation is general enough, that would enable us to require Drop</code> impls to take extra arguments like so:</p> struct </span>Tuna {} </span>struct </span>Cat { </span>name</span>: &</span>'static str </span>} </span>impl </span>Drop </span>for </span>Cat { </span> </span>fn </span>drop</span>(&</span>mut </span>self</span>) with Tuna { </span>// ← Requires `Tuna` to be in scope when invoked </span> println!("</span>{}</span> is snacking on tuna</span>", </span>self</span>.name); </span> } </span>} </span> </span>fn </span>main</span>() { </span> </span>let</span> cat = Cat { name: "</span>Chashu</span>" }; </span> with Tuna {} { </span>// ← Bring `Tuna` into the `with`-scope here </span> </span>drop</span>(cat); </span>// ← Prints: "Chashu is snacking on tuna" </span> } </span>} </span></code></pre> That should work pretty well for types which want to require specific arguments when dropped. Because if Tuna</code> would not be in scope, the type Cat</code> could not be dropped. Now where this falls short is if we want to require destructors to return values too. This is not a MAY return values type of deal, but a MUST return values. If returning a value was not a hard requirement, then we could just add some method and use Drop</code> for all other case. But for true "undroppable types", it must be impossible for a type to be dropped. That means: the only way to destruct the value should be by calling the method on it. It should be possible to express this by leveraging private constructors and contexts like so:</p> struct </span>Priv { </span>_priv</span>: () } </span>// ← Type can only be constructed in this module </span>struct </span>Loaf { </span>_priv</span>: () } </span>// ← Type can only be constructed in this module </span> </span>struct </span>Cat { </span>name</span>: &</span>'static str </span>} </span>impl </span>Cat { </span> </span>// Consume `Cat` and returns a sleepy little loaf </span> </span>fn </span>nap</span>(</span>self</span>) -> Loaf { </span> with Priv { _priv: () } { </span>// ← Create a context to drop `Cat` </span> </span>drop</span>(</span>self</span>); </span>// ← Drop the `Cat` instance </span> Loaf { _priv: () } </span>// ← Return some type </span> } </span> } </span>} </span> </span>/// The type `Priv` cannot be externally constructed, meaning this </span>/// `Drop` impl prevents the type from being dropped outside of the </span>/// `Cat::nap` method. </span>impl </span>Drop </span>for </span>Cat { </span> </span>fn </span>drop</span>(&</span>mut </span>self</span>) with Priv {} </span>} </span> </span>fn </span>main</span>() { </span> </span>let</span> cat = Cat { name: "</span>Chashu</span>" }; </span> cat.</span>nap</span>(); </span>// ✅ Compiles as expected. </span> </span> </span>let</span> cat = Cat { name: "</span>Nori</span>" }; </span> </span>drop</span>(cat); </span>// 💥 Compiler: Cannot invoke `impl Drop for Cat`, `Priv` is not in scope. </span>} </span></code></pre> I think this is pretty fun: a more general feature with wide applicability could be leveraged to solve a more niche feature with more limited applications. I bet if custom diagnostics are generalized enough, we could even improve the error message here, pointing people to call Cat::nap</code> when they can't just drop the type.</p> Of course there are a number of other practical applications that come with undroppable types. The main one being the fact that both any function may panic, and panic currently equates recoverable control flow. For anyone to even attempt using undroppable types outside of a demo scenario, they'd have to grapple with that. But luckily: that again is something that can be resolved independently from this. I'm happy to be able to show that "undroppable types" does not need to be its own, standalone feature, but instead can be expressed using a more general system 2</a></sup>.</p> ^2</sup>For anyone wondering to themselves: "If undroppable types are more restrictive than unleakable types, can't we just use undroppable types everywhere instead?"</em> - The answer is, "Not practically, no"</em>. Unleakable types add minor restrictions to the language (e.g. no mem::forget</code>, no Arc</code>, etc.). Undroppable types add additional major restrictions (e.g. no closure captures, no holding values across ?</code>). Some limitations are acceptable, too many limitations make the experience painful. For more on this, read my 2023 post: Linearity and Control</a>.</p> </div> Tasks are the wrong abstraction Sat, 27 Apr 2024 00:00:00 +0000 Okay, so you've probably read the title and are thinking: "Hey is this clickbait?"</em> - and no dear reader, it is not. I'm fairly convinced at this point that tasks, as an abstraction, are not right for Rust and we would do better to achieve parallel execution through composition. However when it comes to task scheduling</em> things seem less clear, and I would like to have more data available. Because task scheduling strategies determine which APIs we can provide, and as we're looking to standardize the async Rust ecosystem, this will end up being important.</p> So okay, big claim - "Tasks are not the wrong abstraction"</em>. Tasks are a staple in async Rust (and I should know), and task spawning has been around pretty much since the beginning. However, two relatively recent developments have made me start to question whether tasks themselves actually carry their weight, both for single-threaded concurrent workloads and parallel multi-threaded workloads:</p> Both Bytedance's (TikTok's) monoio</code> crate</a> and the glommio</code> crate</a> use a thread-per-core architecture and appear to scale significantly better than Tokio's work-stealing architecture on networking benchmarks</a>. This raises questions about tasks as primitives for parallel async execution.</strong> (edit: I mean "questions" literally here - these are interesting claims which we should research more closely. We get into that more later on in this post).</li> Single-threaded executors introduce 'static</code> lifetimes, often do not correctly propagate errors and cancellation, and in recent testing have shown to perform up to 2-3x worse</a> than their structured counterparts. We have alternative APIs available for concurrency which do not have these issues. This raises questions about tasks as primitives for concurrent async execution.</strong></li> </ol> In this post I want to discuss task spawning for just concurrent but also parallel execution. I want to show some of the issues both of these approaches run into, show how we can do better, and talk about what we need more data on. Finally I want to speculate a little about where we could go with async parallelism and concurrency in Rust. But to save everyone some reading, here's some code roughly summarizing the first half of this post 1</a></sup>:</p> ^{1</sup> Note that the third example here says: "structured". What I actually mean is: as structured as is possible when working within the current limitations of the language. But this API would be entirely structured if we didn't have those limitations.</p> </div>}// concurrent execution of two futures today (structured) </span>let</span> a = async { </span>1 </span>}; </span>// ← does nothing until .awaited </span>let</span> b = async { </span>2 </span>}; </span>// ← does nothing until .awaited </span>let </span>(a, b) = (a, b).</span>join</span>().await; </span>// ← concurrent `.await` </span> </span>// parallel execution of two futures today (unstructured) </span>let</span> a = async_std::spawn(async { </span>1 </span>}); </span>// ← begins parallel execution when created </span>let</span> b = async_std::spawn(async { </span>2 </span>}); </span>// ← begins parallel execution when created </span>let</span> a = a.await; </span>// ← await order does not affect execution </span>let</span> b = b.await; </span>// ← await order does not affect execution </span> </span>// parallel execution of two futures as proposed (structured) </span>let</span> a = async { </span>1 </span>}.</span>par</span>(); </span>// ← does nothing until .awaited </span>let</span> b = async { </span>2 </span>}.</span>par</span>(); </span>// ← does nothing until .awaited </span>let </span>(a, b) = (a, b).</span>join</span>().await; </span>// ← parallel `.await` </span></code></pre> Preface: concurrent and parallel execution</h2> Note (2024-04-27): this section was added after repeated questions about the difference between parallelism and concurrency.</em></p> In order to make sense of this post, it's important to understand the differences between parallelism and concurrency, as well as parallel execution and concurrent execution. These are related but distinct terms, and it can take some time to internalize. My favorite definition of the differences between these comes from Aaron Turon's PhD thesis</a>:</p> Concurrency is a system-structuring mechanism, parallelism is a resource.</p> </blockquote> Put concretely: "concurrency" refers to the way we schedule work. While "parallelism" refers to e.g. the amount of cores a computer has. If we want to perform parallel execution of work</em>, we have to schedule work concurrently using the system's resources for parallelism. We can plot the relationship of parallelism and concurrency in a 2x2 table:</p> </th> no parallelism</th> has parallelism</th></tr></thead> sequential scheduling</strong></td> sequential execution</td> sequential execution</td></tr> concurrent scheduling</strong></td> concurrent execution</td> parallel execution</td></tr> </tbody></table> This table is probably going to surprise some folks. What we're seeing here is that even if we use multiple threads, it's still possible to achieve sequential</em> execution. How can that be? Well dear reader, imagine some exclusive resource shared between N threads. In order for any thread to make progress they must take an exclusive lock out on that resource. That would certainly make use of multiple threads; but execution would be entirely sequential - only one thread can make progress at any given moment. In order to achieve parallel execution</em> it's not enough to just make use of parallelism</em>; we also have to schedule concurrently.</p> Conversely it's also possible to schedule work concurrently despite not having access to any parallelism. An example of this is for example: race two timers with each other. We're waiting on both at the same time, despite not having multiple threads. This is an example of concurrent execution</em> without any access to resources for parallelism.</p> Tasks are the wrong abstraction for parallel async execution</h2> Cq. 2019 I helped popularize the reasoning that: "Tasks should function like the async/.await</code> equivalent of threads"</em>. Among other things that meant that in the runtime</code> crate and subsequently async-std</code> we made sure that tasks always returned JoinHandle</code>s, and that those handles could be awaited to obtain values. Prior to that it was common to manually create async one-shot channels to obtain values from tasks (src</a> 2</a></sup>):</p> ^{2</sup> Romio and Juliex were the reactor and executor Boats and I worked on between 2018-2019. It's a pretty good snapshot of what the state of the art of APIs looked like back then.</p> </div>}//! What using tasks was like cq. December 2018 </span> </span>use </span>std::thread; </span>use </span>futures::{executor, channel::oneshot}; </span> </span>fn </span>main</span>() { </span> </span>let mut</span> handles = vec![]; </span> </span> </span>for </span>_ in </span>0</span>..</span>10 </span>{ </span> </span>let </span>(sender, receiver) = oneshot::channel(); </span> handles.</span>push</span>(receiver); </span> juliex::spawn(async </span>move </span>{ </span> </span>let</span> id = thread::current().</span>id</span>(); </span> sender.</span>send</span>(id); </span> }) </span> } </span> </span> </span>for</span> handler in handles { </span> </span>let</span> id = handler.await; </span>// this was actually `await!` then. </span> println!("</span>handler returned from thread: </span>{id:?}</span>"); </span> } </span>} </span></code></pre> That rationale that "tasks are like async threads"</em> has stuck around, and I think it is wrong. See, concurrency and parallelism in async Rust are different than in non-async Rust. The Thread</code> abstraction packages both concurrency</em> and parallelism</em> into a single abstraction. Whereas in async Rust the two can be decoupled: we can execute any number of futures concurrently, and we don't need to also make us of parallelism for it. Let's walk through some examples, starting with parallel execution using unstructured thread handles:</p> use </span>std::thread; </span> </span>let mut</span> handles = vec![]; </span>handles.</span>push</span>(thread::spawn(\|\| { </span> </span>1 </span>// ← the result of some computation </span>})); </span>handles.</span>push</span>(thread::spawn(\|\| { </span> </span>2 </span>// ← the result of another computation </span>})); </span> </span>let</span> output = handles.</span>into_iter</span>().</span>map</span>(\|</span>h</span>\| h.</span>join</span>().</span>unwrap</span>()).</span>sum</span>(); </span>assert_eq!(output, </span>3</span>); </span></code></pre> Rust does not provide us with a way to say that no, we don't actually want to leverage parallelism here - we just want concurrency. That's why thread::spawn</code> always takes a + Send</code> bound on the closure. In async Rust however, we can just choose to execute work concurrently via the Join</code> family of APIs. Here's an example using futures-concurrency</code>:</p> use </span>futures_concurrency::prelude::; </span> </span>let mut</span> handles = vec![]; </span>handles.</span>push</span>(async { </span> </span>1 </span>// ← the result of some computation </span>})); </span>handles.</span>push</span>(async { </span> </span>2 </span>// ← the result of another computation </span>})); </span> </span>let</span> output = handles.</span>join</span>().await.</span>into_iter</span>().</span>sum</span>(); </span>assert_eq!(output, </span>3</span>); </span></code></pre> Structurally this is very similar to the unstructured threads example; however because futures are lazy and automatically propagate cancellation, they can be considered structured 3</a></sup>. Though typically we'd probably write this example like this instead:</p> ^{3</sup> I've gone into details about the limitations of structured concurrency in Rust today in other posts. TLDR: we need an async version of Drop</code> to actually solve all cases.</p> </div>}use </span>futures_concurrency::prelude::; </span> </span>let</span> a = async { </span>1 </span>}; </span>let</span> b = async { </span>2 </span>}; </span> </span>let</span> output = (a, b).</span>join</span>().await.</span>into_iter</span>().</span>sum</span>(); </span>// ← executes futures concurrently </span>assert_eq!(output, </span>3</span>); </span></code></pre> Now what about parallelism? Well, the point I'm trying to make is that we can achieve parallel execution through composition</em> rather than defining new APIs. It's common practice today to resort to task::spawn</code> APIs for this, mirroring the thread::spawn</code> APIs:</p> use </span>async_std::task; </span> </span>let mut</span> handles = vec![]; </span>handles.</span>push</span>(task::spawn(async { </span> </span>1 </span>// ← the result of some computation </span>})); </span>handles.</span>push</span>(task::spawn(async { </span> </span>2 </span>// ← the result of another computation </span>})); </span> </span>let</span> output = handles.</span>into_iter</span>().</span>map</span>(\|</span>h</span>\| h.await).</span>sum</span>(); </span>assert_eq!(output, </span>3</span>); </span></code></pre> There's a pretty noticeable difference between the previous two examples: one family of async APIs for concurrency, and another family of APIs for both concurrency and parallelism. My pitch here is different: I believe the right way to achieve parallel execution is through composition</em>. What we need is not another way to schedule async work; what we need is a way to define a parallelizable future</em>. And that's something I've prototyped in my tasky</code> crate</a>:</p> use </span>futures_concurrency::prelude::; </span>use </span>tasky::prelude::; </span> </span>let</span> a = async { </span>1 </span>}.</span>par</span>(); </span>// ← added `.par` to create a `ParallelFuture` </span>let</span> b = async { </span>2 </span>}.</span>par</span>(); </span>// ← added `.par` to create a `ParallelFuture` </span> </span>let</span> output = (a, b).</span>join</span>().await.</span>into_iter</span>().</span>sum</span>(); </span>// ← executes two futures in parallel </span>assert_eq!(output, </span>3</span>); </span></code></pre> This approach makes it so we have one way of scheduling concurrent execution, and resources themselves are responsible for deciding whether they should be parallelizable or not. Again: async Rust allows us to decouple parallelism from concurrency in ways not possible in non-async Rust; and so we should design our APIs in ways which leverage that decoupling.</strong></p> With ParallelizableFuture</code> work doesn't start until it is first .await</code>ed. This makes it behave just like any other future. Unlike task handles you can't just fire and forget it; you have to be actively .await</code>ing it to make forward progress. That means a value is always returned, and cancellation will always propagate. And once we have async destructors, those should be able to naturally propagate through the .await</code> points too. This is an API which should be familiar to use, but hard to misuse. It's setting people up for success when it comes to things like propagating cancellation and learning about async concurrency.</p> Tasks are the wrong abstraction for concurrent async execution</h2> This point is probably easier to argue than the previous one: using spawn</code> APIs for just concurrency without also leveraging parallelism is generally not a great experience. Consider the following example, using task::spawn_local</code>:</p> use </span>async_std::task; </span> </span>let mut</span> handles = vec![]; </span>handles.</span>push</span>(task::spawn_local(async { </span> </span>1 </span>// ← the result of some computation </span>})); </span>handles.</span>push</span>(task::spawn_local(async { </span> </span>2 </span>// ← the result of another computation </span>})); </span> </span>let</span> output = handles.</span>into_iter</span>().</span>map</span>(\|</span>h</span>\| h.await).</span>sum</span>(); </span>assert_eq!(output, </span>3</span>); </span></code></pre> This now does the exact same thing as our earlier Join</code> example, except it needs to allocate space on the heap to store each individual future. That's a flat performance tax each task needs to pay; and in this case we've already shown it's avoidable in ever scenario. But that's just performance; there are additional restrictions with regards to ergonomics. The signature of spawn_local</code> is as follows:</p> pub fn </span>spawn_local</span><F, T>(</span>future</span>: F) -> JoinHandle<T> </span>where </span> F: Future<Output = T> + </span>'static</span>, </span> T: </span>'static</span>, </span></code></pre> The 'static</code> lifetime ensures that the future cannot contain any borrows, and resolving it takes a lot of effort because it isn't natural to the language. An example of this is the moro</code> crate, which provides an API for "scoped single-threaded tasks" via an async_scope!</code> macro. The macro is necessary because the lifetimes required for this currently can't be expressed in the language. Here is an adaptation of the thread::scope</code> example</a> converted to use moro</code>:</p> let mut</span> container = vec![</span>1</span>, </span>2</span>, </span>3</span>]; </span>let mut</span> num = </span>0</span>; </span> </span>moro::async_scope!(\|</span>s</span>\| { </span> s.</span>spawn</span>(async { </span> println!("</span>hello from the first scoped thread</span>"); </span> dbg!(&container); </span> }); </span> s.</span>spawn</span>(async { </span> println!("</span>hello from the second scoped thread</span>"); </span> num += container[</span>0</span>] + container[</span>2</span>]; </span> }); </span> println!("</span>hello from the main thread</span>"); </span>}).await; </span> </span>container.</span>push</span>(</span>4</span>); </span>assert_eq!(num, container.</span>len</span>()); </span></code></pre> Let's rewrite this using the futures-concurrency</code>, which doesn't rely on tasks, doesn't enforce 'static</code> lifetimes, and so in turn can freely express what is being expressed here:</p> use </span>futures_concurrency::prelude::; </span> </span>let mut</span> container = vec![</span>1</span>, </span>2</span>, </span>3</span>]; </span>let mut</span> num = </span>0</span>; </span> </span>let</span> a = async { </span> println!("</span>hello from the first future</span>"); </span> dbg!(&container); </span>}; </span> </span>let</span> b = async { </span> println!("</span>hello from the second future</span>"); </span> num += container[</span>0</span>] + container[</span>2</span>]; </span>}; </span> </span>println!("</span>hello from the main future</span>"); </span>let </span>_ = (a, b).</span>join</span>().await; </span>container.</span>push</span>(</span>4</span>); </span>assert_eq!(num, container.</span>len</span>()); </span></code></pre> There are more complex cases possible where we have dynamically updating sets of futures or streams we want to append to, which we want to manage as a group. Getting into that here would mean we'd run long, but for an example of the problem I'd like to point to Niko's mini-redis post</a>, and for an example of how to solve this without tasks or select!</code>, see the second example of the StreamGroup</code> type</a>.</p> I realize that by this point we've veered pretty far off the original point of this section. But it's pretty trivial it probably bears repeating: Tasks</code> with their 'static</code> bounds and performance overhead seem like a poor fit when used solely for concurrency.</strong> And while crates like moro</code> can help overcome some of those challenges, they don't do it fully and don't appear to provide additional expressivity.</p> Stealing and sending</h2> Baked into Rust's async design is the assumption that work-stealing schedulers represent the pinnacle of performance, and therefor it for example makes sense that Waker</code> must always be Send</code>. Work-stealing allows threads to "steal" work from other threads when they're idle. In case one thread has a lot of work scheduled, and another thread is free, this is supposed to enable lower latencies and more throughput.</p> This not without downsides though: in order to facilitate this, it requires that every future contained within a task is Send</code>. The premise of work-stealing is that the performance gains it provides are more than the performance penalties we incur from requiring all futures are Send</code>.</strong> Because making futures Send</code> not only carries a degree of complexity for the language, it also comes with inherent performance penalties because it requires synchronization. You know how you can't use Rc</code> with async/.await</code> - that's a direct artifact of work-stealing designs.</p> The Glommio and Monoio runtimes put this premise into question. Neither of them provide a work-stealing runtime, preferring to use a "thread-per-core" design instead. But by doing this, they do not require to use additional synchronization primitives, and seem to perform better</a> on networked benchmarks than work-stealing runtimes. Monoio claims 2x the throughput of Tokio with 4 cores, and 3x the throughput of Tokio with 16 cores.</strong> This is possible because of their thread-per-core design, but likely also usage of io_uring</code>. I believe we should get updated numbers on this, at least comparing Monoio to the tokio-uring</code> project.</p> Sharded send bounds</h2> Tmandry raised an interesting idea a while back: using Rc</code> and other !Send</code> types inside of Send</code> futures should fine, as long as we can guarantee that all references are moved together as a group. Over in Swift conversations have recently begun about a feature called Region Based Isolation</a>, describing a very similar idea based on the ideas from: "A Flexible Type System for Fearless Concurrency"</a> from PLDI 2022. The Swift SE describes it as follows:</p> Through the usage of isolation regions, the language can prove that transferring a non-Sendable value over an isolation boundary cannot result in races because the value (and any other value that might reference it) is not used in the caller after the point of transfer.</p> </blockquote> Translating this to Rust, I believe it would allow us to do the following:</p> let</span> rc = Rc::new(</span>12</span>usize</span>); </span>// ← `!Send` type </span>task::spawn(async </span>move </span>{ </span>// ← crossing a `Send` boundary </span> dbg!(rc); </span>// ← all references moved: compiler says OK </span>}).await; </span></code></pre> It makes sense that this is all fine: all references to Rc</code> are moved to a new thread, so that's not an issue. But I don't know whether this can hold for all !Send</code> types. I don't think it would be, because occasionally threads will be "special" and so moving a !Send</code> type to another thread even with all references might end up with trouble. This likely would require an additional modifier to Send</code>, integrated through the libraries and possibly language. It's an interesting idea that needs more research before we can consider it viable, but I wanted to make sure to mention it because it does hold promise.</p> parallel futures and send bounds</h2> Both Monoio and Glommio provide a local executor as part of their API. Earlier on in this post I've explained why "local executors" are not a great abstraction. That's why for wasi-async-runtime</code></a>, which also features a single-threaded runtime, I've chosen not to provide a spawn</code> API at all. Instead people are encouraged to use libraries such as futures-concurrency</code> instead.</p> However, even if work-stealing might not carry its weight in Rust right now, I think we might still be able to provide a spawn</code> API. That could potentially even make thread-per-core architectures nicer to express in certain cases where we want an async fn main</code>. The choice of whether we believe work-stealing carries its weight will end up deciding what the right API will be to go with.</strong></p> The task::spawn</code> API for both Tokio and async-std is quite similar and looks something like this:</p> pub fn </span>spawn</span><F, T>(</span>future</span>: F) -> JoinHandle<T> </span>where </span> F: Future<Output = T> + Send + </span>'static</span>, </span> T: Send + </span>'static</span>, </span></code></pre> It takes a Future</code>, and that Future</code> must be Send</code>. Easy, right? Now what if we compare that with the signature of thread::spawn</code>, which looks like this:</p> pub fn </span>spawn</span><F, T>(</span>f</span>: F) -> JoinHandle<T> </span>where </span> F: FnOnce() -> T + Send + </span>'static</span>, </span> T: Send + </span>'static</span>, </span></code></pre> Here we don't have a future, but instead we have a closure. The closure itself is Send</code>, and the value T</code> is Send</code>. At a glance this might look equivalent to the task::spawn</code> APIs, but it's not. This becomes clearer if we encode the original API not as taking Future</code>, but instead as taking IntoFuture</code>:</p> pub fn </span>spawn</span><F, T>(</span>into_future</span>: F) -> JoinHandle<T> </span>where </span> F: IntoFuture<Output = T> + Send + </span>'static</span>, </span> <F as IntoFuture>:IntoFuture: Send + </span>'static</span>, </span> T: Send + </span>'static</span>, </span></code></pre> Not only is the thing we pass into the other thread Send</code>; because we want to allow the ongoing computation itself to be movable between threads the future itself must also be Send</code>. thread::spawn</code> can use !Send</code> types freely because once the computation has started, it will not be moved. If we wanted to encode that, we could do that by changing it to the following signature:</p> pub fn </span>spawn</span><F, T>(</span>into_future</span>: F) -> JoinHandle<T> </span>where </span> F: IntoFuture<Output = T> + Send + </span>'static</span>, </span> T: Send + </span>'static</span>, </span> </span>// `F:IntoFuture` is no longer required to be `Send` here </span></code></pre> Whether task::spawn</code> should take Send</code> bounds is a decision we're going to have to make if we want to encode a standardized spawn API in the stdlib. It's not necessarily zero-sum though; we could probably encode both. But so far evidence seems to indicate that if we want maximum performance thread-per-core is a better approach; while if we want maximum convenience enabling !Send</code> types to work inside of parallelizable futures may actually be simpler to use. More data here would certainly be helpful.</p> A vision for concurrency and parallelism in Rust</h2> So far I've asked a number of questions, and probably observed a little too much. I imagine the average async Rust user will be somewhere between confusion and morbid curiosity of where I'm going with all of this. I want to put a rough sketch forward of where I, Yosh, would like concurrency and parallelism in Rust to eventually get to. I believe we could get pretty far if we made concurrency and parallelism first-class concepts in Rust via two new keywords, which we'll call par</code> and co</code>.</p> Take the venerable Rayon ParallelIterator</code> trait. It allows us to iterate over items in parallel rather than in sequence. While it works great using combinator APIs; it does not allow us to use for</code>-loops the way we'd expect. What if we could do that by introducing for par..in</code> loops:</p> fn </span>square</span>(</span>input</span>: impl Iterator<Item = </span>i32</span>>) -> impl Iterator<Item = </span>i32</span>> { </span> gen </span>move </span>{ </span>// ← current unstable `gen` block notation </span> </span>for</span> par num in input { </span>// ← parallel iteration syntax (hypothetical) </span> </span>yield</span> num num; </span> } </span> } </span>} </span></code></pre> The body of the loop here would be roughly equivalent to Rayon's ParallelIterator::for_each</code> - with the exception that it doesn't just loop but returns an iterator. In async Rust we don't yet have async iteration, but we're currently looking at something like this:</p> fn </span>square</span>(</span>input</span>: impl async Iterator<Item = </span>i32</span>>) -> impl async Iterator<Item = </span>i32</span>> { </span> async gen </span>move </span>{ </span>// ← current unstable `async gen` block notation </span> </span>for</span> await num in input { </span>// ← sequential async iteration syntax (likely) </span> </span>yield</span> num * num; </span> } </span> } </span>} </span></code></pre> This is all unstable and unconfirmed, but it seems likely things will end up along these lines. The least certain part is the exact shapes and names of traits, but that's not the important bit here. Now as we've said we can decouple concurrency and parallelism in async Rust. So what would a concurrent version of this loop look like? Well, one of the main benefits of async Rust is that we can run things concurrently - so what if we made that a first-class part of the language? That's what a hypothetical co</code> keyword could be for:</p> fn </span>square</span>(</span>input</span>: impl async Iterator<Item = </span>i32</span>>) -> impl async Iterator<Item = </span>i32</span>> { </span> async gen </span>move </span>{ </span> </span>for</span> co await num in input { </span>// ← concurrent async iteration syntax (hypothetical) </span> </span>yield</span> num * num; </span> } </span> } </span>} </span></code></pre> Whether we'd spell this co.await</code>, co_await</code> or co await</code> doesn't particularly matter. Making concurrency easier to leverage seems like a nice thing. In terms of implementation we could leverage my recent work on ConcurrentStream</code></a> for this. If we then wanted to extend this to parallelism too, we could instead use par await</code>:</p> fn </span>square</span>(</span>input</span>: impl async Iterator<Item = </span>i32</span>>) -> impl async Iterator<Item = </span>i32</span>> { </span> async gen </span>move </span>{ </span> </span>for</span> par await num in input { </span>// ← parallel async iteration syntax (hypothetical) </span> </span>yield</span> num * num; </span> } </span> } </span>} </span></code></pre> This doesn't just need to stop at iterators either; we could integrate this into futures and async/.await</code> too. Not too different from how Swift's async let</code> notation works. Remember our earlier example using futures-concurrency</code> to evaluate futures concurrently?</p> use </span>futures_concurrency::prelude::; </span> </span>let</span> a = async { </span>1 </span>}; </span>let</span> b = async { </span>2 </span>}; </span> </span>let</span> output = (a, b).</span>join</span>().await.</span>into_iter</span>().</span>sum</span>(); </span>// ← executes futures concurrently </span>assert_eq!(output, </span>3</span>); </span></code></pre> It would be nice if the compiler could perform control-flow analysis directly, and for automatically schedule concurrent execution of futures where permitted, as long as they were called with .co.await</code>:</p> let</span> a = async { </span>1 </span>}.co.await; </span>// ← concurrent await syntax (hypothetical) </span>let</span> b = async { </span>2 </span>}.co.await; </span>// ← concurrent await syntax (hypothetical) </span>assert_eq!(a + b, </span>3</span>); </span></code></pre> And what about parallelism? Well, we should also be able to compose .par</code> with .await</code> to achieve parallel async execution:</p> let</span> a = async { </span>1 </span>}.par.await; </span>// ← parallel await syntax (hypothetical) </span>let</span> b = async { </span>2 </span>}.par.await; </span>// ← parallel await syntax (hypothetical) </span>assert_eq!(a + b, </span>3</span>); </span></code></pre> One of the main appeals of representing async operations as types is that we can then arbitrarily combine them with other futures to achieve concurrent execution. Neither future here needs to be 'static</code> to work with par</code>, and borrows should just work as expected. If we are able to bake concurrent and parallel execution directly into the language, we no longer have to represent the computation as a type. By making .par</code> a modifier to .await</code>, parallel futures would not be represented in the type system and we would be able to solve the scoped parallel async execution problem directly from the language.</strong></p> Oh and the other nice bonus of this: it would work perfectly with #[maybe(async)</code>]</a> function bodies, as we can always fall back from concurrent semantics in async contexts to sequential semantics in non-async contexts. There is probably also something interesting to be said about bare par {}</code> blocks and scoped threads, but that's out of scope (૮꒰ ˶• ༝ •˶꒱ა) for now.</p> Conclusion</h2> In this post we've jumped around a fair bit. You would be forgiven for having some trouble following all threads and ideas. But let me try and summarize the arguments I've attempted to make:</p> Tasks are a poor fit for non-parallel concurrent execution.</strong> They come with additional performance overhead and impose 'static</code> lifetime restrictions, creating knock-on problems.</li> Tasks are a poor fit for parallel concurrent execution.</strong> They are presently designed to function as "async/.await" versions of threads. And as a result combine both concurrency and parallelism into a single API. Instead we would do better to provide a ParallelFuture</code> type which provides parallel execution through composition</em> with concurrency primitives.</li> The success of the Monoio and Glommio runtimes are putting into question whether work-stealing executors are the right fit.</strong> They show significant performance improvements over work-stealing by leveraging a thread-per-core architecture. We need more data to understand the differences before we can make decisions.</li> Deciding between thread-per-core and work-stealing executors has ramifications for what spawn</code> APIs should look like.</strong> Work-stealing executors can't capture !Send</code> types, while non-work-stealing executors can operate on !Send</code> types.</li> There might be benefits to elevating concurrency and parallelism primitives from the libraries into the language.</strong> Fast, safe, concurrent execution is Rust's flagship feature, and making that more accessible would likely pay dividends. This would also provide an alternative way of expressing "scoped tasks".</li> </ol> I hope that if you take anything away from this post it's some of the ground truths of async Rust may be less set in stone than you expected them to be. There definitely appear to be tradeoffs between the various task scheduling approaches and designs, and I don't think we should just assume that work-stealing is the better approach. What we really need is a better understanding of the tradeoffs involved, and for that we're going to need data. Deciding without measuring is not going to work.</p> Oh also: we cannot provide standardized async APIs for parallelism without first stabilizing async destructors.</strong> I find myself repeating this so often I feel like a walking meme at this point. Without async destructors so much of async Rust is broken, and we cannot stabilize interfaces before we know how async destructors will interact with them. Async parallelism specifically is also extra-broken</a>. Async Rust is the flagship feature of priority for Rust as a language, and async closures + async destructors is where we should be spending our time as they're fundamental building blocks.</p> And on a closing note: I just want to put out there that we should dare to dream beyond the mere ossification of the status quo. Better things are possible, as long as we take care of each other and are willing to put in the work. I regularly tell myself this; and now I'm it to you too.</p> Shipping Jco 1.0, WASI 0.2 Wed, 03 Apr 2024 00:00:00 +0000 On this blog I usually post about technical explorations in programming languages, API design, asynchronous computing, and so on. More recently I've re-started my work on WebAssembly, WASI, and their integration with Rust. I expect that work to eventually become more technical again; but my involvement there for the past year has mostly been focused on process and people. And I wanted to take a brief moment to talk about it, because that's the kind of work that's often easily overlooked. And I'd like to take a little victory lap for a second (as a treat).</p> I can proudly say: I helped ship WASI 0.2</a>! Not just a part of it, but I helped make sure we worked through the last key blocker. And mostly on time at that 1</a></sup>! And I'm super proud of it! The entire project was an incredible team effort - it's hard to understate how hard everyone involved has worked on this. And around five years in the making too. My involvement with the actual design and implementation was minimal, and I was mostly involved in the second half of that period. But I can say I've helped in the following ways:</p> ^{1</sup> When first asked to provide estimates I said something along the lines of: "I expect we'll be able to ship in late November at the soonest, but most likely December. December will be tricky because of holidays, but I don't think we'll slip past January."</em> And Jco ended up being ready in mid-January, with WASI 0.2 shipping on January 25th. I think that counts as mostly</em> on time, right?</p> </div>} Helped ship Jco 1.0</strong>: Jco is a native JavaScript WebAssembly toolchain and runtime built for Wasm Components and WASI 0.2. I wrote a blog post about what it is and how to use it on the Bytecode Alliance blog</a>. I basically stepped into the role of PM on that project, ensuring we had clearly defined acceptance criteria to be able to say: "this thing truthfully implements WASI 0.2". And then ended up helping make sure we ended up meeting those criteria. This was my first time donning the PM hat on someone else's project. But that's what was needed, so I was happy to take that on.</li> Helped ship WASI 0.2</strong>: WASI 0.2 is a rebase of the WebAssembly System Interface onto the WebAssembly Components specification. You can read the announcement post here</a>. Wasmtime had full support for WASI 0.2, I want to say, around September 2023. But in order to ratify the actual specification it needed at least two conforming implementations. My JS days are long behind me; but Jco was the best candidate to ship a second conforming implementation. So I figured I'd do what needed to be done, and stepped up to help make sure it shipped. And in the same meeting we announced Jco had started meeting the conformance criteria, the WASI subgroup committee voted to ratify the WASI 0.2 specification.</li> Made the case for a basic async model</strong>: One of the biggest limitations of WASI 0.1 was that it wasn't really suited to write networked applications with. I think it was in 2022 that I started making the case that WASI 0.2 probably should ship with some</em> way to support asynchronous sockets if we wanted people to actually use it for server programming. And we shouldn't wait for a future WASI 0.3 to add that potentially years down the line. And that's what we actually ended up shipping: WASI 0.2 supports async sockets and async HTTP via a poll-based interface</a>, making it possible to write asynchronous networked applications today</em> 2</a></sup>.</li> Helped ship two new Rust backends</strong>: "new" should be in quotation marks here - one of the backends is basically a rename. But yeah; I helped ship support for the renamed wasm32-wasip1</code> target, and the newly introduced wasm32-wasip2</code> targets. My main contributions here were not so much around the implementation, but mostly around writing change proposals, talking to people to help explain the changes and provide assurances, and navigating various organizational challenges. But it's happened, and bar any unexpected issues they should be on the stable channels in a few weeks time! Expect a blog post about this with details on the Rust internals blog soon. (I'm currently procrastinating finishing that one by writing this instead - sorry everyone.)</li> Supported people</strong>: Usually I wouldn't make mention of this as a noteworthy contribution. But there's talking with people and there's talking</em> with people. And over the past two years I've spent a non-trivial amount of time talking with people, listening to what they had to say, and just trying to be there for them. Wasm and WASI are incredibly ambitious projects. I'd roughly describe the work as: imagine being tasked with building several new compilers, defining multiple new operating systems, creating a mix of novel programming language(s) and a shared ABI. Then take all of that and make sure it performs at near-native speeds, is able to be compiled to by virtually every existing programming language, and is usable by non-experts. Then base the entire thing on a capability-based security model and ensure it is perfectly sandboxed by default. Finally, all of that technical work gets compacted into a pressure cooker together with corporate politics, tightening deadlines, and the pressures of working in open source and open standards. Pandemic. Layoffs. Burnout. I hope it's not hard to see why that ends up being A Lot. If there's any takeaway here: sometimes the best open source contribution anyone can make is just to sit and listen to what people have to say.</li> </ul> ^2</sup>WASI 0.3's async model</a> is going to be an enormous step up from the current status quo though - and timing-wise the work on that seems to be developing faster than people anticipated. However back in 2022, I was worried that if WASI 0.2 didn't ship with async support, there would be a chance adoption would be too slow to make it to WASI 0.3. The state now is that key industry use cases are unlocked by WASI 0.2, and WASI 0.3 is mainly about making that work better and faster. And at least to me so far this seems to have been the right approach.</p> </div> And that's it. Unfortunately no cool type theory bits in this post (I like those a lot; I will</em> talk about type systems for hours if left unchecked) - but I still wanted to share a little bit about my other recent work. I often catch myself in the mindset that if I'm not directly coding or designing features, it doesn't actually count as having worked on something. But that's probably unfair to myself: I ended up spending several hundred hours doing useful work to help ship WASI 0.2. And I believe I've actually managed to move the needle on getting WASI in people's hands sooner. Doing things I wasn't necessarily the most comfortable with, but it ended up working out! And while I'm not usually one to gloat, I felt like maybe, perhaps, I do get to gloat a little about this.</p> I'd also be remiss if I didn't point out that the folks in both the WASI subgroup committee and in the Bytecode Alliance are such a pleasure to work with. People regularly talk about how smart they are, and the quality of the work they produce - but I'd like to instead emphasize just how kind and caring they are. Talented people, unafraid to dream big, who care about both their work and each other. WASI 0.2 and Wasm Components are incredibly impressive, and over the next 6-12 months we're going start seeing the first results of the impact it will have in the space of applied computing.3</a></sup></p> ^3</sup>To provide an early taste: this headline</a> will probably read like marketing hypeware to most people; some company you've likely never heard of claiming order-of-magnitude improvements in a mature technology. But the technical work underpinning it is legit. And while I'm not in a position to vet the exact claims made, they don't at all seem out of line with what I've seen WASI being able to do so far. I expect similar headlines are going to keep following now that WASI 0.2 has been ratified and is starting to be deployed to production. Exciting times!</p> </div> Designing an Async Runtime for WASI 0.2 Thu, 29 Feb 2024 00:00:00 +0000 In 2019 Stjepan Glavina and I developed the async-std</code> runtime</a>. That was an off-shoot from the runtime</code> project</a>, which itself was an attempt to make it easier to abstract over different async runtimes. One of the things I'm most proud of in the work I did on async-std</code> is the core IO abstraction which Stjepan later factored out into the polling</code></a> and async-io</code></a> libraries as part of the smol</code> project</a>, where he took them beyond my initial prototype code into robust building blocks which can work on their own.</p> Anyway, that little stroll down memory lane serves a purpose: I just finished building Yet Another Async runtime</a>. Not to tell people to actually go and use this - but intended to be more like a working, minimal, but also correct implementation of an async runtime for WASI 0.2</a>. The purpose of this post is to detail how I built it, so you can build your own (if you want to). I'm one of the first to write this code, and maybe even the first to write a dedicated runtime, which means that if Smol</a>, Monoio</a>, Glommio</a>, or Tokio</a> want to add support for WASI 0.2 they're going to have to implement what I already implemented. So I figured I might save folks some trouble, and document the work I just finished doing.</p> The WASI 0.2 polling model</h2> WASI 0.2 is readiness-based rather than completion-based. That means we wait for the host system to tell us we're ready to take an action, rather than wait for the host system that an action has successfully completed (completion-based). WASI 0.3 will likely switch to a completion-based system because Linux io_uring</code></a> and Windows' ioringapi</code></a> are completion-based and perform really well, but we don't have that yet.</p> The core polling system in WASI consists of two components: the Pollable</code> type</a> and the poll</code> function</a>. The Pollable</code> type represents "interest" in an event. Say we had some kind of read</code> call; there would be an associated Pollable</code> for that call, which we could submit to the host to say: "please let me know when there's a reason for me to call read</code>". This is the "readiness" part of the system - you submit your interest in an operation to the host system, and then the host system will periodically yield a list of which operations are good to be called. The way to schedule that interest is via the poll</code> function.</p> A key difference between the epoll</code> and WASI 0.2's poll</code> model is that in WASI we don't schedule interest in resources</em> (e.g. file descriptors), but we schedule interest in concrete operations. Under epoll</code> we'd tell the kernel something like: "hey, here's an fd representing some socket - I'm interested anytime I can read or write new data to it". In WASI we're more precise; we make a specific method call, which returns a type. That type will have some way to get the underlying data, but also a method subscribe</code> which returns a pollable. We're then supposed to wait for the poll</code> call to tell us that the Pollable</code> is ready - and then we can call the method to get the underlying data without it returning an error. That's probably a lot of words, so here's a basic example:</p> use </span>wasi::http::outgoing_handler::{handle, OutgoingRequest}; </span>use </span>wasi::http::types::{Fields, Method, Scheme}; </span>use </span>wasi::io::poll; </span> </span>fn </span>main</span>() { </span> </span>// Construct an HTTP request </span> </span>let</span> fields = Fields::new(); </span> </span>let</span> req = OutgoingRequest::new(fields); </span> req.</span>set_method</span>(&Method::Get).</span>unwrap</span>(); </span> req.</span>set_scheme</span>(Some(&Scheme::Https)).</span>unwrap</span>(); </span> req.</span>set_path_with_query</span>(Some("</span>/</span>")).</span>unwrap</span>(); </span> req.</span>set_authority</span>(Some("</span>example.com</span>")).</span>unwrap</span>(); </span> </span> </span>// Send the request and wait for it to complete </span> </span>let</span> res = </span>handle</span>(req, None).</span>unwrap</span>(); </span>// 1. We're ready to send the request over the network </span> </span>let</span> pollable = res.</span>subscribe</span>(); </span>// 2. We obtain the `Pollable` from the response future </span> poll::poll(&[&pollable]); </span>// 3. Block until we're ready to look at the response </span> </span> </span>// We're now ready to try and access the response headers. If </span> </span>// the request was unsuccessful we might still get an error here, </span> </span>// but we won't get an error that we tried to read data before the </span> </span>// operation was completed. </span> </span>let</span> res = res.</span>get</span>().</span>unwrap</span>().</span>unwrap</span>().</span>unwrap</span>(); </span> </span>for </span>(name, _) in res.</span>headers</span>().</span>entries</span>() { </span> println!("</span>header: </span>{name}</span>"); </span> } </span>} </span></code></pre> When run via cargo-component</a> (and the -- -S http</code> flag is passed), this should print the following:</p> header: age </span>header: cache-control </span>header: content-type </span>header: date </span>header: etag </span>header: expires </span>header: last-modified </span>header: server </span>header: vary </span>header: x-cache </span>header: content-length </span></code></pre> Designing the poller abstraction</h2> In the HTTP request example we've shown how to make a single HTTP request and registered that one operation with the poll</code> call. One of the main benefits of async computing is the ability to perform ad-hoc concurrency</a>: we don't just want to be able to wait for the one operation to complete; we want to be able to wait on any number of operations to complete at the same time</em>. This is why the poll</code> function takes a list of Pollable</code> types, and returns a list of which ones are ready to continue doing work.</p> For that we need to create some type which can hold the pollables for us, be somewhat efficient about it, and allow us to associate the list of "ready" events with some notion of identity. This is important because we're building a runtime for async Rust, which requires that we associate "readiness events" with "calls to specific wakers</a>". We can begin by defining a struct containing a slab</a>. This is an efficient key-value data structure which on entry gives you a key to access the value. It is unordered and reuses allocations - which is great because we'll be continuously registering and deregistering interest in different Pollable</code>s.</p> #[</span>derive</span>(Debug)] </span>pub</span>(</span>crate</span>) </span>struct </span>Poller { </span> </span>pub</span>(</span>crate</span>) </span>targets</span>: Slab<Pollable>, </span>} </span></code></pre> Whenever we insert a type into Slab</code> it returns a key of type usize</code>. To make working with it easier, and make bridging into the WASI world easier, we're going to define our own EventKey</code> type to wrap the keys for us, like so:</p> /// A key representing an entry into the poller </span>#[</span>repr</span>(transparent)] </span>#[</span>derive</span>(Debug, PartialEq, Eq, PartialOrd, Ord, Hash, Clone, Copy)] </span>pub</span>(</span>crate</span>) </span>struct </span>EventKey(pub(crate) </span>u32</span>); </span></code></pre> Now we're ready to implement the base CRUD methods on the Poller</code>: new</code>, insert</code>, remove</code>, and get</code>. Aside from the translation to EventKey</code>, there's nothing interesting going on here.</p> impl </span>Poller { </span> </span>/// Create a new instance of `Poller` </span> </span>pub</span>(</span>crate</span>) </span>fn </span>new</span>() -> </span>Self </span>{ </span> </span>Self </span>{ </span> targets: Slab::new() </span> } </span> } </span> </span> </span>/// Insert a new `Pollable` target into `Poller` </span> </span>pub</span>(</span>crate</span>) </span>fn </span>insert</span>(&</span>mut </span>self</span>, </span>target</span>: Pollable) -> EventKey { </span> </span>let</span> key = </span>self</span>.targets.</span>insert</span>(target); </span> EventKey(key as </span>u32</span>) </span> } </span> </span> </span>/// Get a `Pollable` if it exists. </span> </span>pub</span>(</span>crate</span>) </span>fn </span>get</span>(&</span>self</span>, </span>key</span>: &EventKey) -> Option<&Pollable> { </span> </span>self</span>.targets.</span>get</span>(key.</span>0 </span>as </span>usize</span>) </span> } </span> </span> </span>/// Remove an instance of `Pollable` from `Poller`. </span> </span>/// </span> </span>/// Returns `None` if no entry was found for `key`. </span> </span>pub</span>(</span>crate</span>) </span>fn </span>remove</span>(&</span>mut </span>self</span>, </span>key</span>: EventKey) -> Option<Pollable> { </span> </span>self</span>.targets.</span>try_remove</span>(key.</span>0 </span>as </span>usize</span>) </span> } </span>} </span></code></pre> And finally we're ready to implement the most interesting part: implementing the method to wait for events in the Poller</code>, and map them to their respective EventKey</code>s. When we submit a list of Pollable</code>s to the poll</code> call, we get a list of indexes</em> back which refer to the indexes of the pollables we submitted. These are not the same values as the EventKey</code>s we have, so we have to construct a lookup table to map the indexes of the pollables back to the event keys.</p> impl </span>Poller { </span> </span>/// Block the current thread until a new event has triggered. </span> </span>/// </span> </span>/// This will clear the value of `ready_list`. </span> </span>pub</span>(</span>crate</span>) </span>fn </span>block_until</span>(&</span>mut </span>self</span>) -> Vec<EventKey> { </span> </span>// We're about to wait for a number of pollables. When they wake we get </span> </span>// the indexes* back for the pollables whose events were available - so </span> </span>// we need to be able to associate the index with the right waker. </span> </span> </span>// We start by iterating over the pollables, and keeping note of which </span> </span>// pollable belongs to which waker index </span> </span>let mut</span> indexes = Vec::with_capacity(</span>self</span>.targets.</span>len</span>()); </span> </span>let mut</span> targets = Vec::with_capacity(</span>self</span>.targets.</span>len</span>()); </span> </span>for </span>(index, target) in </span>self</span>.targets.</span>iter</span>() { </span> indexes.</span>push</span>(index); </span> targets.</span>push</span>(target); </span> } </span> </span> </span>// Now that we have that association, we're ready to poll our targets. </span> </span>// This will block until an event has completed. </span> </span>let</span> ready_indexes = </span>poll</span>(&targets); </span> </span> </span>// Once we have the indexes for which pollables are available, we need </span> </span>// to convert it back to the right keys for the wakers. Earlier we </span> </span>// established a positional index -> waker key relationship, so we can </span> </span>// go right ahead and perform a lookup there. </span> ready_indexes </span> .</span>into_iter</span>() </span> .</span>map</span>(\|</span>index</span>\| EventKey(indexes[index as </span>usize</span>] as </span>u32</span>)) </span> .</span>collect</span>() </span> } </span>} </span></code></pre> And with that, our Pollable</code> abstraction is complete!</p> Designing the reactor abstraction</h2> Now that we can track lists of pollables and a way to submit them - it's time to bring wakers into the mix and create a type which can be used by individual futures to register interest in events. The WASI async runtime is fundamentally single-threaded, so what we'll want to use here are just the Rc</code> and RefCell</code> types. We can now construct our data types; again starting with the core structures:</p> use super</span>::polling::{EventKey, Poller}; </span> </span>use </span>std::collections::HashMap; </span>use </span>std::task::Poll; </span>use </span>std::task::Waker; </span>use </span>std::{cell::RefCell, rc::Rc}; </span>use </span>wasi::io::poll::Pollable; </span> </span>/// Manage async system resources for WASI 0.1 </span>#[</span>derive</span>(Debug, Clone)] </span>pub struct </span>Reactor { </span> </span>inner</span>: Rc<RefCell<InnerReactor>>, </span>} </span> </span>/// The private, internal `Reactor` implementation - factored out so we can take </span>/// a lock of the whole. </span>#[</span>derive</span>(Debug)] </span>struct </span>InnerReactor { </span> </span>poller</span>: Poller, </span> </span>wakers</span>: HashMap<EventKey, Waker>, </span>} </span> </span>impl </span>Reactor { </span> </span>/// Create a new instance of `Reactor` </span> </span>pub</span>(</span>crate</span>) </span>fn </span>new</span>() -> </span>Self </span>{ </span> </span>Self </span>{ </span> inner: Rc::new(RefCell::new(InnerReactor { </span> poller: Poller::new(), </span> wakers: HashMap::new(), </span> })), </span> } </span> } </span>} </span></code></pre> The reason we can get away with Rc<RefCell<Inner>></code> is because we control all the accesses in the methods. We guarantee they are always short-lived and will never overlap. Which in turn provides us with a convenient way to to be able to call methods on &self</code> rather than &mut self</code>. Speaking of which, time to implement the first method - which is a way to register interest in a pollable</code>.</p> Registering interest in a pollable</h3> This provides an async method wait_for</code> which allows our reactor to wait asynchronously until the call related to the pollable</code> is ready to be performed.</p> impl </span>Reactor { </span> </span>/// Wait for the pollable to resolve. </span> </span>pub</span> async </span>fn </span>wait_for</span>(&</span>self</span>, </span>pollable</span>: Pollable) { </span> </span>let mut</span> pollable = Some(pollable); </span> </span>let mut</span> key = None; </span> </span> </span>// This function is the core loop of our function; it will be called </span> </span>// multiple times as the future is resolving. </span> future::poll_fn(\|</span>cx</span>\| { </span> </span>// Start by taking a lock on the reactor. This is single-threaded </span> </span>// and short-lived, so it will never be contended. </span> </span>let mut</span> reactor = </span>self</span>.inner.</span>borrow_mut</span>(); </span> </span> </span>// Schedule interest in the `pollable` on the first iteration. On </span> </span>// every iteration, register the waker with the reactor. </span> </span>let</span> key = key.</span>get_or_insert_with</span>(\|\| reactor.poller.</span>insert</span>(pollable.</span>take</span>().</span>unwrap</span>())); </span> reactor.wakers.</span>insert</span>(key, cx.</span>waker</span>().</span>clone</span>()); </span> </span> </span>// Check whether we're ready or need to keep waiting. If we're </span> </span>// ready, we clean up after ourselves. </span> </span>if</span> reactor.poller.</span>get</span>(key).</span>unwrap</span>().</span>ready</span>() { </span> reactor.poller.</span>remove</span>(key); </span> reactor.wakers.</span>remove</span>(key); </span> Poll::Ready(()) </span> } </span>else </span>{ </span> Poll::Pending </span> } </span> }) </span> .await </span> } </span>} </span></code></pre> What's nice about this is that unlike other runtimes, where interest is registered on the object</em>, here we merely register interest on an operation</em>. That means that "wait for this operation to be ready" is a matter of calling reactor.wait_for(pollable).await</code>. That is significantly different than the juggling of syscalls required in other polling models; and it means we can encode all of the invariants necessary for the reactor directly as part of the reactor.</p> In recent conversations in the Rust WG Async we've been discussing whether it makes sense to have some kind of reactor hook as part of the Context</code> in future - or whether perhaps other mechanisms would be better. From this example I believe we can conclude that while it might be possible</em> to share a reactor through the future's context - at a high level that will always be mapped the same way to the underlying function calls. Which at least to me puts into question whether that is the most natural mapping.</p> Blocking until events are ready</h3> With that out of the way, we're ready to implement the wrapper around our Poller::block_until</code> call. As we implemented, it will wait for all registered pollers</code>, and give us back a list of EventKey</code>s for the ones which are ready to be woken. All we have to do is iterate over the list of keys, and call each related waker:</p> /// Block until new events are ready. Calls the respective wakers once done. </span>pub</span>(</span>crate</span>) </span>fn </span>block_until</span>(&</span>self</span>) { </span> </span>let mut</span> reactor = </span>self</span>.inner.</span>borrow_mut</span>(); </span> </span>for</span> key in reactor.poller.</span>block_until</span>() { </span> </span>match</span> reactor.wakers.</span>get</span>(&key) { </span> Some(waker) => waker.</span>wake_by_ref</span>(), </span> None => panic!("</span>tried to wake the waker for non-existent `{key:?}`</span>"), </span> } </span> } </span>} </span></code></pre> At first glance it might seem silly that this goes through the motions of calling the wakers. WASI 0.2 is currently single-threaded, so by default it will always just wake the one waker. However, it is common and encouraged to use wakers to distinguish between events. Concurrency primitives may construct their own wakers to keep track of identity and wake more precisely. We do not control the wakers constructed by libraries, and it is for this reason that we have to call all the wakers - even if by default they will do nothing.</p> Designing the block_on abstraction</h2> Now that we have our reactor done, we need a way to drive it. For that we're going to define a function block_on</code> which takes a closure returning a future and gives it access to the Reactor</code>. As long as the future returned by the closure is live, we'll keep making progress on it - waiting on the reactor each time the future returns Pending</code>.</p> We'll be driving futures by hand here, so the first step is to define our own instance of Waker</code>. We're not supporting any form of spawn</code> function, and instead rely on users to leverage libraries for structured concurrency, so out of the box we can just make it a noop.</p> /// Construct a new no-op waker </span>// NOTE: we can remove this once <https://github.com/rust-lang/rust/issues/98286> lands </span>fn </span>noop_waker</span>() -> Waker { </span> </span>const </span>VTABLE</span>: RawWakerVTable = RawWakerVTable::new( </span> </span>// Cloning just returns a new no-op raw waker </span> \|_\| </span>RAW</span>, </span> </span>// `wake` does nothing </span> \|_\| {}, </span> </span>// `wake_by_ref` does nothing </span> \|_\| {}, </span> </span>// Dropping does nothing as we don't allocate anything </span> \|_\| {}, </span> ); </span> </span>const </span>RAW</span>: RawWaker = RawWaker::new(ptr::null(), &</span>VTABLE</span>); </span> </span> </span>// SAFETY: all fields are no-ops, so this is safe </span> </span>unsafe </span>{ Waker::from_raw(</span>RAW</span>) } </span>} </span></code></pre> Next up is the actual block_on</code> implementation. The core logic is basically a loop { match {} }</code> statement which just calls reactor.block_until</code> on each iteration until the future completes.</p> </span>/// Start the event loop </span>pub fn </span>block_on</span><F, Fut>(</span>f</span>: F) -> </span>Fut::</span>Output </span>where </span> F: FnOnce(Reactor) -> Fut, </span> Fut: Future, </span>{ </span> </span>// Construct the reactor </span> </span>let</span> reactor = Reactor::new(); </span> </span> </span>// Create the future and pin it so it can be polled </span> </span>let</span> fut = (f)(reactor.</span>clone</span>()); </span> </span>let mut</span> fut = pin!(fut); </span> </span> </span>// Create a new context to be passed to the future. </span> </span>let</span> waker = </span>noop_waker</span>(); </span> </span>let mut</span> cx = Context::from_waker(&waker); </span> </span> </span>// Either the future completes and we return, or some IO is happening </span> </span>// and we wait. </span> </span>loop </span>{ </span> </span>match</span> fut.</span>as_mut</span>().</span>poll</span>(&</span>mut</span> cx) { </span> Poll::Ready(res) => </span>return</span> res, </span> Poll::Pending => reactor.</span>block_until</span>(), </span> } </span> } </span>} </span></code></pre> And with that, our async runtime is complete!</p> Using the async runtime</h2> This section was added post-publication on 2024-03-07</em></p> Now that we have our runtime, we can use it to wrap the calls from the wasi</code> library and make them async. For example, say we wanted to have a non-blocking sleep</code> function which waits for some Duration</code>. We could write that by using the subscribe_duration</code></a> function and connecting it to the reactor. All we need to do is wrap it, call it, and then wait on the pollable. That should be simple enough to do with what we've written:</p> use </span>std::time::Duration; </span>use </span>wasi::time::monotonic_clock::subscribe_duration; </span> </span>pub</span> async </span>fn </span>sleep</span>(</span>duration</span>: Duration, </span>reactor</span>: &Reactor) { </span> </span>let</span> duration = duration.</span>as_nanos</span>() as </span>u64</span>; </span>// 1. Convert the duration to nanosecond resolution </span> </span>let</span> pollable = </span>subscribe_duration</span>(duration); </span>// 2. Obtain the pollable </span> reactor.</span>wait_for</span>(pollable).await; </span>// 3. Wait for the pollable to be ready </span>} </span></code></pre> It should be simple enough to take APIs from the wasi</code> crate, and pair them with a call to reactor.wait_for(..).await</code>. That's the base pattern we's use to asyncify basically the entire wasi</code> API surface.</p> On the absence of a local executor</h2> WASI 0.2 is single-threaded, and fundamentally doesn't need access to a task.spawn</code> abstraction. In sync Rust we use threads to combine parallelism + concurrency; but in async Rust we can separate the two. The futures-concurrency</a> library provides access to any mode of concurrency you might want; meaning that in the absence of parallelism there is no reason for an executor to exist.</p> Instead APIs such as Vec::join</code></a> and StreamGroup</code></a> even provide access to the unbounded concurrency primitives people typically use "local executors" for - but without any of the scoped lifetime issues plaguing those abstractions. Here is an example of how to concurrently make two separate HTTP requests without having to rely on "local tasks":</p> fn </span>main</span>() { </span> </span>block_on</span>(\|</span>reactor</span>\| async { </span> </span>let</span> client = Client::new(reactor); </span>// <- using an unpublished wrapper around `wasi::http` </span> </span> </span>let</span> a = async { </span> </span>let</span> url = "</span>https://example.com</span>".</span>parse</span>().</span>unwrap</span>(); </span> </span>let</span> req = Request::new(Method::Get, url); </span> </span>let</span> res = client.</span>send</span>(req).await; </span> </span> </span>let</span> body = </span>read_to_end</span>(res).await; </span> </span>let</span> body = String::from_utf8(body).</span>unwrap</span>(); </span> println!("</span>{body}</span>"); </span> }; </span> </span> </span>let</span> b = async { </span> </span>let</span> url = "</span>https://example.com</span>".</span>parse</span>().</span>unwrap</span>(); </span> </span>let</span> req = Request::new(Method::Get, url); </span> </span>let</span> res = client.</span>send</span>(req).await; </span> </span> </span>let</span> body = </span>read_to_end</span>(res).await; </span> </span>let</span> body = String::from_utf8(body).</span>unwrap</span>(); </span> println!("</span>{body}</span>"); </span> }; </span> </span> (a, b).</span>join</span>().await; </span>// concurrently await both `a` and `b`. </span> }) </span>} </span></code></pre> Conclusion</h2> In this post I've explained WASI's polling model and showed step-by-step how to use it to build your own async runtime. I hope this will be useful for maintainers of async Rust runtimes, as well as hobbyists wanting to go through the motions of writing their own. If you prefer to just use the code I've shared here today, you can do so by installing the wasi-async-runtime</code> crate</a>.</p> Extending Rust's Effect System Fri, 09 Feb 2024 00:00:00 +0000 </iframe> This is the transcript of my RustConf 2023 talk: "Extending Rust's Effect System", presented on September 13th 2023 in Albuquerque, New Mexico and streamed online.</em></p> Introduction</h2> Rust has continuously evolved since version 1.0 was released in 2015. We've added major features such as the try operator (?</code>), const generics, generic associated types (GATs), and of course: async/.await</code>. Out of those four features, three are what can be considered to be "effects". And though we've been working on them for a long time, they are all still very much in-progress.</p> Hello, my name is Yosh and I work as a Developer Advocate for Rust at Microsoft. I've been working on Rust itself for the past five years, and I'm among other things a member of the Rust Async WG, and the co-lead of the Rust Effects Initiative.</p> The thesis of this talk is that we've unknowingly shipped an effect system as part of the language in since Rust 1.0. We've since begun adding a number of new effects, and in order to finish integrating them into the language we need support for effect generics</em>.</p> In this talk I'll explain what effects are, what makes them challenging to integrate into the language, and how we can overcome those challenges.</p> Rust Without Generics</h2> When I was new to Rust it took me a minute to figure out how to use generics. I was used to writing JavaScript, and we don’t have generics there. So I found myself mostly writing functions which operated on concrete types</em>. I remember my code felt pretty clumsy, and it wasn't a great experience. Not compared to, say, the code the stdlib provides.</p> An example of a generic stdlib function is the io::copy</code> function. It reads bytes from a reader, and copies them into a writer. We can give it a file, a socket, or any combination of the two, and it will happily copy bytes from one into the other. This all works as long as we give it the right types.</p> But what if Rust actually didn't have generics? What if the Rust I used to write at the beginning was actually all we had? How would we write this io::copy</code> function? Well, given we're trying to copy bytes between sockets and file types, we could probably hand-code individual functions for these. For our two types here we could write four unique functions.</p> But unfortunately for us that would only solve the problem right in front of us. But the stdlib doesn’t just have two types which implement read and write. It has 18 types which implement read, and 27 types which implement write. So if we wanted to cover the entire API space of the stdlib, we’d need 486 functions in total. And if that was the only way we could implement io::copy</code>, that would make for a pretty bad language.</p> Now luckily Rust does have generics, and all we ever need is the one copy</code> function. This means we're free to keep adding more types into the stdlib without having to worry about implementing more functions. We just have the one copy</code> function, and the compiler will take care of generating the right code for any types we give it.</p> Why effect generics?</h2> Types are not the only things in Rust we want to be generic over. We also have "const generics" which allow functions to be generic over constant values. As well as "value generics" which allow functions to be generic over different values. This is how we can write functions which can take different values - which is a feature present in most programming languages.</p> fn </span>by_value</span>(</span>cat</span>: Cat) { .. } </span>fn </span>by_reference</span>(</span>cat</span>: &Cat) { .. } </span>fn </span>by_mutable_reference</span>(</span>cat</span>: &</span>mut</span> Cat) { .. } </span></code></pre> But not everything that can lead to API duplication are things we can be generic over. For example, it's pretty common to create different methods or types depending on whether we take a value as owned, as a reference, or as a mutable reference. We also often create duplicate APIs for constant values and for runtime values. As well as create duplicate structures depending on whether the API needs to be thread-safe or not.</p> But out of everything which can lead to API duplication, effects are probably one of the biggest ones. When I talk about effects in Rust, what I mean is certain keywords such as async/.await</code> and const</code>; but also ?</code>, and types such as Result</code>, and Option</code>. All of these have a deep, semantic connection to the language, and changes the meaning of our code in ways that other keywords and types don't.</p> Sometimes we'll write code which doesn't have the right effects, leading to effect mismatches</em>. This is also known as the function coloring problem</em>, as described by Robert Nystrom. Once you become aware of effect mismatches</em> you start seeing them all over the place, not just in Rust either.</p> The result of these effect mismatches is that using effects in Rust essentially drops you into a second-rate experience. Whether you're using const, async, Result, or Error - almost certainly somewhere along the line you'll run into a compatibility issue.</p> let</span> db: Option<Database> = ..; </span>let</span> db = db.</span>filter</span>(\|</span>db</span>\| db.name? == "</span>chashu</span>"); </span></code></pre> Take for example the Option::filter</code> API. It takes a type by reference and returns a bool. If we try and use the ?</code> operator inside of it we get an error, because the function doesn't return Result</code> or Option</code>. Not being able to use ?</code> inside of arbitrary closures is an example of an effect mismatch.</p> But simple functions like that only scratch the surface. Effect mismatches are present in almost every single trait in the stdlib too. Take for example something common like the Debug</code> trait which is implemented on almost every type in Rust.</p> We could implement the Debug</code> trait for our made-up type Cat</code>. The parameter f</code> here implements io::Write</code> and represents a stream of bytes. And using the write!</code> macro we can write bytes into that stream. But if for some reason we wanted to write bytes asynchronously into, say, an async socket. Well, we can't do that. fn fmt</code> is not an async function, which means we can't await inside of it.</p> One way out of this could be to create some kind of intermediate buffer, and synchronously write data into it. We could then write data out of that buffer asynchronously. But that would involve extra copies we didn't have before.</p> If we wanted to make it identical to what we did before, the solution would be to create a new</em> AsyncDebug</code> trait which can</em> write data asynchronously into the stream. But we now have duplicate traits, and that's exactly the problem we're trying to prevent.</p> It's tempting to say that maybe we should just add the AsyncDebug</code> trait and call it a day. We can then also add async versions of Read</code>, Write</code>, and Iterator</code> too. And perhaps Hash</code> as well, since it too writes to an output stream. And what about From</code> and Into</code>? Perhaps Fn</code>, FnOnce</code>, FnMut</code>, and Drop</code> too since they're built-ins? And so on. The reality is that effect mismatches are structural, and duplicating the API surface for every effect mismatch leads to an exponential explosion of APIs. Which is similar to what we've seen with data type generics earlier on.</p> Let me try and illustrate this for a second. Say we took the existing family of Fn</code> traits and introduced effectful versions of them. That is: versions which work with unsafe</code> 1</a></sup>, async</code>, try</code>, const</code>, and generators. With one effect we're up to six unique traits. With two effects we're up to twelve. With all five we're suddenly looking at 96 different traits.</p> ^{1</sup> Correction from 2024: after having discussed this with Ralf Jung we've concluded that semantically unsafe</code> in Rust is not an effect. But syntactically it would be fair to say that unsafe</code> is "effect-like". As such any notion of "maybe-unsafe" would be nonsensical. We don't discuss such a feature in this talk, but it is worth clearing up ahead of time in case this leaves people wondering.</p> </div> The problem space in the stdlib is really broad. From analyzing the Rust 1.70 stdlib, by my estimate about 75% of the stdlib would interact with the const effect. Around 65% would interact with the async effect. And around 30% would interact with the try effect. The exact numbers are imprecise because parts of the various effects are still in-progress. How much this will result in practice, very much will depend on how we end up designing the language.</p> If you compare the numbers then it appears that close to 100% of the stdlib would interact with one or more effect. And about 50% would interact with two or more effects. If we consider that whenever effects interact with each other they can lead to exponential blowup, this should warn us that clever one-off solutions won't cut it. I believe that the best way to deal with this is to instead allow Rust to enable items to be generic over effects.</p> Stage I: Effect-Generic Trait Definitions</h2> Now that we've taken a good look at what happens when we can't be generic over effects, it's time we start talking about what we can do about it. The answer, unsurprisingly, is to introduce effect generics into the language. To cover all uses will take a few steps, so let's start with the first, and arguably most important one: effect-generic trait definitions.</p> This is important because it would allow us to introduce effectful traits as part of the stdlib. Which would among other things would help standardize the various async ecosystems around the stdlib.</p> pub trait </span>Into<T>: Sized { </span> </span>fn </span>into</span>(</span>self</span>) -> T; </span>} </span></code></pre> impl </span>Into<Loaf> </span>for </span>Cat { </span> </span>fn </span>into</span>(</span>self</span>) -> Loaf { </span> </span>self</span>.</span>nap</span>() </span> } </span>} </span></code></pre> Let's use a simple example here: the Into</code> trait. The Into</code> trait is used to convert from one type into another. It is generic over a type T</code>, and has one function "into" which consumes Self</code> and returns the type T</code>. Say we have a type cat which when it takes a nap turns into a cute little loaf. We can implement Into<Loaf></code> for Cat</code> by calling self.nap</code> in the function body.</p> pub trait </span>AsyncInto<T>: Sized { </span> async </span>fn </span>into</span>(</span>self</span>) -> T; </span>} </span></code></pre> impl </span>AsyncInto<Loaf> </span>for </span>Cat { </span> async </span>fn </span>into</span>(</span>self</span>) -> Loaf { </span> </span>self</span>.</span>nap</span>().await </span> } </span>} </span></code></pre> But what if the cat doesn't take a nap straight away? Maybe nap</code> should actually be an async function. In order to await</code> nap inside the trait impl, the into</code> method would need to be async. If we were writing an async trait from scratch, we could do this by exposing a new AsyncInto</code> trait with an async into</code> method.</p> But we don't just want to add a new trait to the stdlib, instead we want to extend</em> the existing Into</code> trait to work with the async effect. The way we could extend the Into</code> trait with the async</code> effect is by making the async effect optional</em>. Rather than requiring that the trait is always sync or async, implementors should be able to choose which version of the trait they want to implement.</p> #[</span>maybe</span>(async)] </span>impl </span>Into<Loaf> </span>for </span>Cat { </span> #[</span>maybe</span>(async)] </span> </span>fn </span>into</span>(</span>self</span>) -> Loaf { </span> </span>self</span>.</span>nap</span>() </span> } </span>} </span></code></pre> The way this would work is by adding a new notation on the trait: "maybe async". We don't yet know what syntax we want to use for "maybe async", so in this talk we'll be using attributes. The way the "maybe async" notation works is that we mark all methods which we want to be "maybe async" as such. And then mark our trait itself as "maybe async" too.</p> impl </span>Into<Loaf> </span>for </span>Cat { </span> </span>fn </span>into</span>(</span>self</span>) -> Loaf { </span> </span>self</span>.</span>nap</span>() </span> } </span>} </span></code></pre> impl </span>async Into<Loaf> for Cat { </span> async </span>fn </span>into</span>(</span>self</span>) -> Loaf { </span> </span>self</span>.</span>nap</span>().await </span> } </span>} </span></code></pre> Implementors then get to choose whether they want to implement the sync or async versions of the trait. And depending on which version they choose, the methods then ends up being either sync or async. This system would be entirely backwards-compatible, because implementing the sync version of Into</code> would remain the same as it is today. But people who want to implement the async version would be able to, just by adding a few extra async</code> keywords to the impl.</p> impl </span>async Into<Loaf> for Cat { </span> async </span>fn </span>into</span>(</span>self</span>) -> Loaf { </span> </span>self</span>.</span>nap</span>().await </span> } </span>} </span></code></pre> impl </span>Into<Loaf, true> </span>for </span>Cat { </span> </span>type </span>ReturnTy = </span>impl </span>Future<Output = Loaf>; </span> fn into(self) -> Self::ReturnTy { </span> async </span>move </span>{ </span> </span>self</span>.</span>nap</span>().await </span> } </span> } </span>} </span></code></pre> Under the hood the implementations desugars to regular Rust code we can already write today. The sync implementation of the type returns a type T</code>. But the async impl returns an impl Future</code> of T</code>. Under the hood it is just a single const bool and some associated types.</p> good diagnostics</li> gradual stabilization,</li> backwards-compatibility</li> clear inference rules</li> </ul> It would be reasonable to ask why we're bothering with a language feature, if the desugaring ends up being so simple. And the reason is: effects are everywhere, and we want to make sure effect generics feel like part of the language. That not only means that we want to tightly control the diagnostics. We also want to enable them to be gradually introduced, have clear language rules, and be backwards-compatible.</p> But if you keep all that in mind, it's probably okay to think of effect generics as mostly syntactic sugar for const bools + associated types.</p> Stage II: Effect-Generic Bounds, Impls, and Types</h2> Being able to declare effect-generic traits is only the beginning. The stdlib not only exposes traits, it also exposes various types and functions. And effect-generic traits don't directly help with that.</p> pub fn </span>copy</span><R, W>( </span> </span>reader</span>: &</span>mut</span> R, </span> </span>writer</span>: &</span>mut</span> W </span>) -> io::Result<()> </span>where </span> R: Read, </span> W: Write; </span></code></pre> Let's take our earlier io::copy</code> example again. As we've said copy</code> takes a reader and writer, and then copies bytes from the reader to the writer. We've seen this.</p> pub fn </span>async_copy</span><R, W>( </span> </span>reader</span>: &</span>mut</span> R, </span> </span>writer</span>: &</span>mut</span> W </span>) -> io::Result<()> </span>where </span> R: AsyncRead, </span> W: AsyncWrite; </span></code></pre> Now what would it look like if we tried adding an async version of this to the stdlib today. Well, we'd need to start by giving it a different name so it doesn't conflict with the existing copy</code> function. The same goes for the trait bounds as well, so instead of taking Read</code> and Write</code>, this function would take AsyncRead</code> and `AsyncWrite.</p> pub fn </span>async_copy</span><R, W>( </span> </span>reader</span>: &</span>mut</span> R, </span> </span>writer</span>: &</span>mut</span> W </span>) -> io::Result<()> </span>where </span> R: async Read, </span> W: async Write; </span></code></pre> Now things get a little better once we have effect-generic trait definitions. Rather than needing to take async duplicates of the Read</code> and Write</code> traits, the function can instead choose the async versions of the existing Read</code> and Write</code> traits. That's already better, but it still means we have two versions of the copy</code> function.</p> #[</span>maybe</span>(async)] </span>pub fn </span>copy</span><R, W>( </span> </span>reader</span>: &</span>mut</span> R, </span> </span>writer</span>: &</span>mut</span> W </span>) -> io::Result<()> </span>where </span> R: #[maybe(async)] Read, </span> W: #[maybe(async)] Write; </span></code></pre> Instead the ideal solution would be to allow copy</code> itself to be generic over the async effect, and make that determine which versions of Read</code> and Write</code> we want. These are what we call "effect-generic bounds". The effect of the function and the effect of the bounds it takes all become the same. In literature this is also known as "row-polymorphism".</p> copy</span>(reader, writer)?; </span>// infer sync </span>copy</span>(reader, writer).await?; </span>// infer async </span>copy::<async>(reader, writer).await?; </span>// force async </span></code></pre> Because the function itself is now generic over the async effect, we need to figure out at the call-site which variant we intended to use. This system will make use of inference</em> to figure it out. That's a fancy way of saying that the compiler is going to make an educated guess about which effects the programmer intended to use. If they used .await</code> they probably wanted the async version. Otherwise they probably wanted the sync version. But as with any guess: sometimes we guess wrong, so for that reason we want to provide an escape hatch by enabling program authors to force the variant. We don't know the exact syntax for this yet, but we assume this would likely be using the turbofish notation.</p> struct </span>File { .. } </span>impl </span>File { </span> </span>fn </span>open</span><P>(</span>p</span>: P) -> Result<</span>Self</span>> </span> </span>where </span> P: AsRef<Path>; </span>} </span></code></pre> But effect-generics aren't just needed for functions. If we want to make the stdlib work well with effects, then types will need effect-generics too. This might seem strange at first, since an "async type" might not be very intuitive. But for example files on Windows need to be initialized as either sync or async. Which means that whether they're async or not isn't just a property of the functions, it's a property of the type.</p> Let's use the stdlib's File</code> type as our example here. For simplicity let's assume it has a single method: open</code> which returns either an error or a file.</p> struct </span>AsyncFile { .. } </span>impl </span>AsyncFile { </span> async </span>fn </span>open</span><P>(</span>p</span>: P) -> Result<</span>Self</span>> </span> </span>where </span> P: AsRef<AsyncPath>; </span>} </span></code></pre> If we wanted to provide an async version of File</code>, we again would need to duplicate our interfaces. That means a new type AsyncFile</code>, which has a new async method open</code>, which takes an async version of Path</code> as an argument. And Path</code> needs to be async because it itself has async filesystem methods on it. As I've said before: once you start looking you notice effects popping up everywhere.</p> #[</span>maybe</span>(async)] </span>struct </span>File { .. } </span> </span>#[</span>maybe</span>(async)] </span>impl </span>File { </span> #[</span>maybe</span>(async)] </span> </span>fn </span>open</span><P>(</span>p</span>: P) -> Result<</span>Self</span>> </span> </span>where </span> P: AsRef<</span>#</span>[maybe(async)] Path>; </span>} </span></code></pre> Instead of creating a second AsyncFile</code> type, with effect generics on types we'd be able to open File</code> as async instead. Allowing us to keep just the one File</code> definition for both sync and async variants.</p> #[</span>maybe</span>(async)] </span>fn </span>copy</span><R, W>(</span>reader</span>: R, </span>writer</span>: W) -> io::Result<()> { </span> </span>let mut</span> buf = vec![</span>4028</span>]; </span> </span>loop </span>{ </span> </span>match</span> reader.</span>read</span>(&</span>mut</span> buf).await? { </span> </span>0 </span>=> </span>return </span>Ok(()), </span> n => writer.</span>write_all</span>(&buf[</span>0</span>..n]).await?, </span> } </span> } </span>} </span></code></pre> Now I've sort of hand-waved away the internal implementations of both the copy</code> function and the File</code> type. The way they work is a little different for the two. In the case of the copy</code> function, the implementation between the async and non-async variants would be identical. If the function is compiled as async, everything works as written. But if the function compiles as sync, then we just remove the .await</code>s and the function should compile as expected.</p> As a result of this "maybe-async" functions can only call sync or other "maybe-async" functions. But that should be fine for most cases.</p> impl </span>File { </span> #[</span>maybe</span>(async)] </span> </span>fn </span>open</span><P>(</span>p</span>: P) -> Result<</span>Self</span>> { </span> </span>if </span>IS_ASYNC </span>{ .. } </span>else </span>{ .. } </span> } </span>} </span></code></pre> Concrete types like File</code> are a little trickier. They often want to run different code depending on which effects it has. Luckily types like File</code> already conditionally compile different code depending on the platform, so introducing new types conditions shouldn't be too big of a jump. The key thing we need is a way to detect in the function body whether code is being compiled as async or not - basically a fancy bool.</p> We can already do this for the const effect using the const_eval_select</code> intrinsic. It's currently unstable and a little verbose, but it works reliably. We should be able to easily adapt it to something similar for async and the rest of the effects too.</p> What are effects?</h2> Systems research on effects has been a topic in computer science for nearly 40 years. That's about as old as C++. It's become a bit of a hot topic recently in PL spheres with research languages such as Koka, Eff, and Frank showing how effects can be useful. And languages such as Scala, and to a lesser extent Swift, adopting effect features.</p> When people talk about effects they will broadly refer to one of two things:</p> Algebraic Effect Types:</strong> which are semantic notations on functions and contexts that grant a permission to do</em> something.</li> Algebraic Effect Handlers:</strong> which are a kind of typed control-flow primitive which allows people to define their own versions of async/.await</code>, try..catch</code>, and yield</code>.</li> </ul> A lot of languages which have effects provide both effect types and effect handlers. These can be used together, but they are in fact distinct features. In this talk we'll only be discussing effect types.</p> pub</span> async </span>fn </span>meow</span>(</span>self</span>) {} </span>pub const unsafe fn </span>meow</span>() {} </span></code></pre> What we've been calling "effects" in this talk so far have in fact been effect types</em>. Rust hasn't historically called them this, and I believe that's probably why effect generics weren't on our radar until recently. But it turns out that reinterpreting some of our keywords as effect types actually makes perfect sense, and provides us with a strong theoretical framework for how to reason about them.</p> We also have unsafe</code> which allows you to call unsafe</code> functions. The unstable try-block feature which doesn't require you to Ok</code>-wrap return types. The unstable generator closure syntax which gives you access to the yield</code> keyword. And of course the const</code> keyword which allows you evaluate code at compile-time.</p> async { </span>async_fn</span>().await }; </span>// async effect </span>unsafe </span>{ </span>unsafe_fn</span>() }; </span>// unsafe effect </span>const </span>{ </span>const_fn</span>() }; </span>// const effect </span>try { </span>try_fn</span>()? }; </span>// try effect (unstable) </span>\|\| { </span>yield</span> my_type }; </span>// generator effect (unstable) </span></code></pre> In Rust we currently have five different effects: async</code>, unsafe</code>, const</code>, try</code>, and generators. All six of these are in various stages of completion. For example: async Rust has functions and blocks, but no iterators or drop. Const doesn't have access to traits yet. Unsafe functions can't be lowered to Fn</code> traits. Try does have the ?</code> operator, but try blocks are unstable. And generators are entirely unstable; we only have the Iterator</code> trait.</p> Some of these effects are what folks on the lang team have started calling "carried". Those are effects which will desugar to an actual type in the type system. For example when you write async fn</code>, the return type will desugar to an impl Future</code>.</p> Some other effects are what we're calling: "uncarried"</em>. These effects don't desugar to any types in the type system, but serve only as a way to communicate information back to the compiler. This is for example const</code> or unsafe</code>. While we do check that the effects are used correctly, they don't end up being lowered to actual types.</p> let</span> x = try async { .. }; </span></code></pre> 1. -> impl Future<Output = Result<T, E>> </span>2. -> Result<impl Future<Output = T>, E> </span></code></pre> When we talk about carried effects, effect composition becomes important. Take for example "async" and "try" together. If we have a function which has both? What should the resulting type be? A future of Result? Or a Result containing a Future?</p> Effects on functions are order-independent sets</em>. While Rust currently does require you declare effects in a specific order, carried effects themselves can only be composed in one way. When we stabilized async/.await</code>, we decided that if an async function returned a Result, that should always return an impl Future</code> of Result</code>. And because effects are sets</em> and not dependent on ordering, we can define the way carried effects should compose as part of the language.</p> People can still opt-out from the built-in composition rules by manually writing function signatures. But this is rare, and for the overwhelming majority of uses the built-in composition rules will be the right choice.</p> const fn </span>meow</span>() {} </span>// maybe-const </span>const </span>{} </span>// always-const </span></code></pre> The const</code> effect is a bit different from the other effects. const</code> blocks are always</em> evaluated during compilation. While const</code> functions merely can</em> be evaluated during during compilation. It's perfectly fine to call them at runtime too. This means that when we write const fn</code>, we're already writing effect-generics. This mechanism is the reason why we've gradually been able to introduce const into the stdlib in a backwards-compatible way.</p> Const is also a bit strange in that among other things it disallows access to the host runtime, it can't allocate, and it can't access globals. This feels different from effects like say, async</code>, which only allow you to do more things.</p> effect set</th> can access</th> cannot access</th></tr></thead> std rust</td> non-termination, unwinding, non-determinism, statics, runtime heap, host APIs</td> N/A</td></tr> alloc</td> non-termination, unwinding, non-determinism, globals, runtime heap</td> host APIs</td></tr> core</td> non-termination, unwinding, non-determinism, globals</td> runtime heap, host APIs</td></tr> const</td> non-termination, unwinding</td> non-determinism, globals, runtime heap, host APIs</td></tr> </tbody></table> What's missing from this picture is that all functions in Rust carry an implicit set of effects. Including some effects we can't directly name yet. When we write const</code> functions, our functions have a different set of effects, than if we write no_std</code> functions, which again are different from regular "std" rust functions.</p> The right way of thinking about const, std, etc. is as adding a different effects to the empty set of effects. If we start from zero, then all effects are merely additive. They just add up to different numbers.</p> Unfortunately in Rust we can't yet name the empty set of effects. In effect theory this is called the "total effect". And some languages such as Koka do support the "total" effect. In fact, Koka's lead developer has estimated that around 70% of a typical Koka program can be total. Which begs the question: if we could express the total effect in Rust, could we see similar numbers?</p> Stage III: More Effects</h2> So far we've only talked about how we could finish the work on existing effects such as const</code> and async</code>. But one nice thing of effect generics is that they would not only allow us to finish our ongoing effects work. It would also lower the cost of introducing new</em> effects to the language.</p> Which opens up the question: if we could add more effects, which effects might make sense to add? The obvious ones would be to actually finish adding try</code> and generator functions. But beyond that, there are some interesting effects we could explore. For brevity I'll only discuss what these features are, and not show code examples.</p> no-divergence</strong>: guarantees that a function cannot loop indefinitely, opening up the ability to perform static runtime-cost analysis.</li> no-panic</strong>: guarantees a function will never produce a panic, causing the function to unwind.</li> parametricity</strong>: guarantees that a function only operates on its arguments. That means no implicit access to statics, no global filesystem, no thread-locals.</li> capability-safety</strong>: guarantees that a function is not only parametric, but can't downcast abstract types either. Say if you get an impl Read</code>, you can't reverse it to obtain a File</code>.</li> destructor linearity</strong>: guarantees that Drop</code> will always</em> be called, making it a safety guarantee.</li> pattern types</strong>: enables functions to operate directly on variants of enums and numbers</li> must-not-move types</strong>: would be a generalization of pinning and the pin-project system, making it a first-class language feature</li> </ul> Though there's nothing inherently stopping us from adding any of these features into Rust today, in order to integrate them into the stdlib without breaking backwards-compatibility we need effect generics first.</p> effect </span>const </span>= diverge + panic; </span>effect core = </span>const </span>+ statics + non_determinism; </span>effect alloc = core + heap; </span>effect std = alloc + host_apis; </span></code></pre> This brings us to the final part of the design space: effect aliases. If we keep adding effects it's very easy to eventually reach into a situation where we have our own version of "public static void main".</p> In order to mitigate that it would instead be great if we could name specific sets of effects. In a way we've already done that, where const</code> represents "may loop forever" and "may panic". If we actually had "may loop forever" and "may panic" as built-in effects, then we could redefine const</code> as an alias to those.</p> Fundamentally this doesn't change anything we've talked about so far. It's just that this would syntactically be a lot more pleasant to work with. So if we ever reach a state where we have effect generics and we want notice we maybe have one too many notation in front of our functions, it may be time for us to start looking into this more seriously.</p> Outro</h2> Rust already includes effect types such as async, const, try, and unsafe. Because we can't be generic over effect types yet, we usually have to choose between either duplicating code, or just not addressing the use case. And this makes for a language which feels incredibly rough once you start using effects. Effect generics provide us with a way to be generic over effects, and we've shown they can be implemented today as mostly as syntax sugar over const-generics.</p> We're currently in the process of formalizing the effect generic work via the A-Mir-Formality. MIR Formality is an in-progress formal model of Rust's type system. Because effect generics are relatively straight forward but have far-reaching consequences for the type system, it is an ideal candidate to test as part of the formal model.</p> In parallel the const WG has also begun refactoring the way const functions are checked in the compiler. In the past const-checking happened right before borrow checking at the MIR level. In the new system const-checking will happen much sooner, at the HIR level. This will not only make the code more maintainable, it will also be generalizable to more effects if needed.</p> Once both the formal modeling and compiler refactorings conclude, we'll begin drafting an RFC for effect-generic trait definitions. We expect this to happen sometime in 2024.</p> And that's the end of this talk. Thank you so much for being with me all the way to the end. None of the work in this talk would have been possible without the following people:</p> Oliver Scherer (AWS)</li> Eric Holk (Microsoft)</li> Niko Matsakis (AWS)</li> Daan Leijen (Microsoft)</li> </ul> Thank you!</p> Will it block? Wed, 07 Feb 2024 00:00:00 +0000 Will this function block?</p> fn </span>work</span>(</span>x</span>: </span>u64</span>, </span>y</span>: </span>u64</span>) -> </span>u64 </span>{ </span> x + y </span>} </span></code></pre> How about this function?</p> fn </span>work</span>(</span>x</span>: </span>u32</span>) -> </span>u32 </span>{ </span> </span>let mut</span> prev = </span>0</span>; </span> </span>let mut</span> curr = </span>1</span>; </span> </span>for </span>_ in </span>0</span>..x { </span> </span>let</span> next = prev + curr; </span> prev = curr; </span> curr = next; </span> } </span> curr </span>} </span></code></pre> Or this one?</p> use </span>sha256::digest; </span> </span>fn </span>work</span>(</span>input</span>: &</span>str</span>) -> String { </span> </span>digest</span>(input) </span>} </span></code></pre> Will this function block?</p> use </span>std::{thread, time::Duration}; </span>fn </span>work</span>() { </span> thread::sleep(Duration::from_nanos(</span>10</span>)); </span>} </span></code></pre> Perhaps this one will?</p> use </span>std::{thread, time::Duration}; </span>fn </span>work</span>() { </span> thread::sleep(Duration::from_millis(</span>10</span>)); </span>} </span></code></pre> Or how about this one, does this block?</p> use </span>std::{thread, time::Duration}; </span>fn </span>work</span>() { </span> thread::sleep(Duration::from_secs(</span>10</span>)); </span>} </span></code></pre> Let's try another; does this block?</p> use </span>std::{thread, time::Duration}; </span>fn </span>work</span>() { </span> thread::spawn(\|\| {}).</span>join</span>(); </span>} </span></code></pre> Up next: does this block?</p> use </span>std::{thread, time::Duration}; </span>fn </span>work</span>() { </span> println!("</span>meow</span>"); </span>} </span></code></pre> Last one; does this block?</p> use </span>std::fs::File; </span>use </span>std::io::prelude::; </span>use </span>std::os::unix::io::FromRawFd; </span> </span>fn </span>work</span>() { </span> </span>let mut</span> f = </span>unsafe </span>{ File::from_raw_fd(</span>1</span>) }; </span> write!(&mut f, "</span>meow</span>").</span>unwrap</span>(); </span>} </span></code></pre> And so on.</p> Determining whether something "is blocking" is a messy endeavor because computers are messy. Whether something blocks is not something we can objectively determine, but depends on our own definitions and uses. A system call which resolves quickly might not be considered "blocking" at all ^{1</a></sup>, while a computation which takes a long time might be. The Tokio folks wrote the following about it a few years ago</a>:</p>} ^{1</sup> Modern storage and networking devices can get pretty darn fast. A modern PCIe connection can transfer 128GB/s or 1Tb/s. Sub-millisecond response times for storage are not unheard of either. And even network calls, for example between processes (containers) or between machines in a data center, can resolve incredibly quickly.</p> </div>} […] it is very hard to define "progress". A naive definition of progress is whether or not a task has been scheduled for over some unit of time. For example, if a worker has been stuck scheduling the same task for more than 100ms, then that worker is flagged as blocked and a new thread is spawned. In this definition, how does one detect scenarios where spawning a new thread reduces throughput? This can happen when the scheduler is generally under load and adding threads would make the situation much worse.</p> </blockquote> Even if we decide something is "blocking" after an arbitrary cutoff - that might still not do us any good if the overall throughput of the system is reduced. Determining whether something is objectively blocking or not might be the most useful question to be asking. Alice Ryhl (also of Tokio) had the following to say</a>:</p> In case you forgot, here's the main thing you need to remember: Async code should never spend a long time without reaching an .await</code>.</p> </blockquote> I believe this is a much more useful framing. Rather than attempting (and failing) to categorize calls as "blocking", we should identify where we're spending longer stretches of time without yielding back to the runtime - and ensure we can correct it when we don't. In async-std I introduced the task::yield_now</code></a> function to help with this 2</a></sup>. Tokio subsequently adopted it</a> 3</a></sup>, and for WASI's async model a similar function is being considered too.</p> ^2</sup>The original issue where I introduced this idea is still available</a> in case anyone is reading about the motivation for this at the time. We very clearly modeled it after thread::yield_now</code>, but async.</p> </div> ^{3</sup> I believe they've been working on an alternative system to this for a few years now, but I'm not sure what the state of that is.</p> </div> The goal of this post is to show by example the ambiguity present when attempting to create an objective classification of which code counts as "blocking". As well as taking the opportunity to highlight previous writing on the topic.</p>}Special thanks to Jim Blandy</a>, who during my review of The Crab Book</a> convinced me it is actually fine to use a synchronous mutex in asynchronous code, in turn pointing me towards Alice's writing</a>.</em></p> Reframing WIT as primarily a machine format Sun, 19 Nov 2023 00:00:00 +0000 This is a "short" post. Unlike my usual (lengthy) research pieces, this is mostly a train-of-thought, low-edit post intended to convey an idea or perspective I thought would be useful to write down.</em></p> edit(2023-11-19):</strong> I've changed the title of this post from "Rethinking WIT" to "Reframing WIT". I strongly hold that WIT being human-readable is a good thing we shouldn't change, even though in this post I make the case that despite that it probably shouldn't be the primary way we interact with it.</em></p> Earlier this year I wrote about how I believe HTML is too complex to be feasibly written by hand</a>, and rather than linting manually authored HTML for errors it's probably more effective to treat HTML as a target which should be compiled to. I ended up putting these ideas to practice in the html crate for Rust</a>, and I think that ended up being roughly the right idea. At the core of this was a reframing of "HTML" as this thing that's meant to be read and written by humans, to a format that is primarily intended to be produced and ingested by machines.</p> Recently I've been using WIT - the IDL format used to define Wasm Components with - and I'm slowly starting to form opinions about it. Yesterday I wrote about why I believe ABIs can be described as: "A type system + object encoding + calling convention"</a>. But today I want to take a slightly different perspective on WIT and share an opinion I've started forming: I believe that WIT should primarily be produced and consumed by tools, and not by humans. This is a departure from how people have been reasoning about WIT so far - which is to think of WIT as a human-readable format people will want to read and write by hand. In this post I want to explain why I believe this is probably a better way to reason about WIT.</p> Interacting with WIT</h2> I believe that from a programmer's perspective all WIT usage can be roughly broken up into two categories: consuming WIT definitions to use in programs. And producing WIT definitions to be used in other programs, or to run your program in a certain runtime. I believe that when we import</em> WIT, we probably don't want to read WIT files directly. What we're more likely interested in are the bindings which would be generated for our programming language. Or perhaps if we're familiar with WIT itself, we might be interested in an interactive doc explorer a la rustdoc</code> to navigate the type hierarchy.</p> If we're going to be exporting code using WIT, then we probably want to start by writing the code itself. In Rust when we want to serialize types we don't typically start writing a JSON Schema</a> - instead we start by writing our structs and adding a serde derive</a> at the top. And we don't just do this for JSON, we take a similar data-first approach</a> for command lines too. I believe Wasm Components and their WIT definitions would likely do well with a similar approach. Because integrating it into the language like this is by far the path of least resistance.</p> I believe this will probably hold true for the majority of programmers. Rather than writing or consuming types authored in WIT, it'll be much easier to reason about components in terms of how they integrate into the language. And if for some reason reasoning about the bindings is not sufficient, a documentation tool will probably be the easier to way to explore WIT APIs directly.</p> Why perspectives may differ</h2> WASI Preview 2 hasn't been released yet, and so up until this point the majority of people working with WIT haven't actually been programmers trying to build things with</em> WIT. So far the people using WIT have mostly been the people working on WASI itself, primarily working to define the language-agnostic interfaces everyone will be writing their programs against.</p> And so I think this is a likely explainer for why the readability of WIT has been highly valued so far. The work the majority of people using WIT today is to write WIT by hand. But I believe that once Preview 2 releases, this will likely become a minority use case. If done right, most people working with WIT will be using it via programming languages instead.</p> Even for system rewrites I believe it's likely easiest to generate bindings for an existing impl directly from a language. Then take the WIT definitions generated by that, and use that as the interface for the new implementation. At no point does this require people to directly interact with WIT definitions.</p> On tradeoffs</h2> To me it makes sense that if we believe that the primary way people use WIT is either write it by hand, or read the source files, that the property of "readability" is one we're least keen to compromise on. I'm not quite sure about the exact tradeoffs; but my understanding is that the "readability" of WIT has been fairly important.</p> So far I've been making the case that I believe people are more likely to be ingesting WIT using language-specific bindings, producing it via language ascriptions, and reading it via documentation tools. And so if that's the case, then "readability" may perhaps be something we're more willing to compromise on in favor of other properties.</p> That's not that I'm trying to argue that being able to write WIT by hand or read it directly isn't important. What I'm trying to say is that I don't believe it necessarily is the most</em> important property. There may be other properties, like expressivity or machine-parsability, which would compromise readability. If we value "readability" differently, then we may very well end up making different decisions.</p> Where to go from here?</h2> I believe that the more closely we can integrate Wasm Components/WASI with language toolchains and guest languages, the easier it will be for people to adopt it. The explainer I've used for this in the past is in terms of "deltas". The less people need to deviate from their existing workflows to do a new thing, the more likely it is they'll want to try the new thing. Applied to Components: the more it feels like importing and exporting Components is like using any other types in that language, the more likely it is that people are willing to use it.</p> With that in mind, I would like us to start thinking about ways in which we can remove manually authored WIT entirely from the end-to-end experience. If done right, both importing and exporting WIT definitions from guest languages should feel completely native. As if we were importing or exporting any other library written in that same language. I feel like we're close to that for some languages 1</a></sup>, but I believe we can probably improve on it.</p> ^{1</sup> I want to add here: the existing efforts I've seen on this front are really impressive! I'm in no way trying to snub the existing work done!</p> </div>}The other thing I think we should explore is a native wit-doc</code> tool which can be used to render WIT definitions to HTML. My thinking is that this should be modeled loosely after rustdoc or Swift's DocC. Given plans are plans to make progress on the Warg registry next year - this seems like something which could integrate nicely with that too. Over in Rust land, one of my least favorite splits is between "docs.rs" and "cargo.io" - in my opinion it would make sense to merge these two. And given wit-bindgen already runs in the browser</a>, we could probably also generate previews of what generated output bindings for various programming languages would look like. Not unlike the SDK example generators used in some of the world's fanciest docs</a>.</p> Conclusion</h2> In this (short) post I've tried to make the case that while historically WIT has been authored by hand and read from source files, that may not necessarily be continue being true in the future. To me it seems more likely that once Preview 2 is released, people using Wasm Components will primarily be interested in using Wasm Components directly from source languages. Since that is the path of least friction. And in the case people need to look up WIT APIs directly, a (hosted) documentation tool will probably provide a better experience than reading files directly from source.</p> I argue that if we no longer believe that people will primarily interact with WIT by manually authoring WIT files, and reading them from source, that the "readability" property of the WIT format may become less important. That's not to say it should ever be deemed unimportant, since there are valid reasons to interact directly with WIT files. But it may be the case that if change how we value this property, we will become more willing to trade it off in favor of other properties.</p> Reasoning about ABIs Sat, 18 Nov 2023 00:00:00 +0000 Note: this post is not my usual research posts; but more a loose collection of thoughts I've been thinking of recently. I might start tagging these as "short", since I'm writing and publishing before fully validating.</em></p> Not long after I started programming I developed an intuition for what APIs are: they are the interfaces we use in our applications to communicate between distinct components. But there was this other term, "ABI", which I didn't quite understand. "Application Binary Interface" is not very descriptive if you haven't used it before. Does it mean "binary" because it's encoded? But APIs use encodings too. If an ABI is separate from an API, are we not supposed to program against ABIs? What are the differences?</p> Over the past few months I've been helping out with getting WASI Preview 2 over the finish line1</a></sup>. Part of WASI Preview 2 are Wasm Components and WIT definitions, which both are part of an overarching system called "The Component Model". In broad terms, Components in Wasm are able to communicate with one another via a stable ABI encoding, which is accompanied by WIT (an Interface Definition Language) that can be used to describe system interfaces. For example like so:</p> ^{1</sup> ETA for WASI Preview 2 at the time of writing: either December this year or January next year; it's November now.</p> </div>}package example:cat; </span> </span>interface cat { </span> meow: func() -> string; </span>} </span> </span>world cats { </span> export cat; </span>} </span></code></pre> This defines a package cat</code> in the namespace example</code>, providing a set of export cats</code>, which includes the cat</code> interface. In the component model we know how to take this interface, and encode this as a Wasm Component. Which can then be imported directly by other components, and have bindings generated for it for any language via wit-bindgen</code></a>. WASI (WebAssembly System/Standard Interfaces) are all defined in terms of WIT, and describe standard ways of interfacing with things like system clocks, filesystems, networks, and in the future caches and queues too.</p> I'm really used to thinking of ABIs in terms of data encodings and byte offsets. In Rust we can slap pub extern "C"</code> on any type to change the way it's laid out in memory. But it has limitations; like for example it doesn't know what to do with async fn</code>. In the past I've also worked with other IDL formats such as WebIDL</a>, COM</a>, and WinMD</a>. And while all of these have helped me understood what an ABI does</em>, they haven't really helped me understand what an ABI fundamentally is</em>. But now that I've worked with WIT, I now believe that the better way to think of ABIs as just a different application of type systems:</p> Application Binary Interface</strong>: A type system with a specified serialization format and calling convention 2</a></sup>.</li> Programming Language</strong>: A type system with runtime behavior (operational semantics</a>).</li> </ul> ^{2</sup> A "calling convention" determines the way functions are invoked; how arguments are mapped to registers, etc.</p> </div>}The Wasm Component model doesn't just define types like "string" or "record", it also has a (limited 3</a></sup>) notion of generic types too. For example result</code> and option</code> are generic types, so you can define functions which return a result<string, error></code> to indicate they're fallible. I never considered that ABIs could contain ADTs 4</a></sup>, but WIT clearly has it. Which to me signals that (with some constraints), we can apply the wealth of type system theory we've built up over the years to ABIs as well.</p> ^{3</sup> There are currently four built-in generics in WIT. It's unclear whether it'll support user-defined generics in the future since that would complicate implementations. I'd love to see it though.</p> </div>}^{4</sup> "Algebraic Data Type" - which are types which can carry data. These include both enums (sum types) and structs (product types) in Rust. What's cool about them is that they're compositional because an ADT can carry more ADTs.</p> </div>}The boundaries between ABIs and programming languages in practice can often get murky though. As we've seen with extern "C"</code> in Rust, programming languages can define ABIs internally as well. And vice versa: some ABIs may depend on runtime semantics of a language too. But in its broadest terms: I believe an ABI must</em> define a data encoding, but doesn't need to define runtime semantics. And a programming language must</em> define operational semantics, but doesn't have to define an encoding 5</a></sup>. And I feel like that provides a much crisper way to think about what ABIs actually are, rather than just what they're used for?</p> ^{5</sup> A data encoding is still necessary to actually encode instructions on a machine; but that doesn't need to be part of the programming language - it can be an implementation of the compiler. And multiple compilers for the same language can truthfully compile the same language, despite not needing to agree on the encoding. As long as the operational semantics end up working as specified.</p> </div>}Two Kinds of ABIs (added 2023-11-18)</h2> I think it's fair to say there are two ways to think about ABIs. One is just as the low-level encoding of types + calling conventions. The WASI Component model has a document dedicated to what is called "the canonical ABI"</a> which only specifies the encoding and calling conventions.</p> The way I'm talking about ABIs here in this post is one step above that, where we not only consider the WASI canonical ABI, but also the WIT IDL format as a part of "ABI". Both are designed with each other in mind, and having WIT without the canonical ABI doesn't make much sense. This feels very similar to my experience working on windows-rs</a>, which ingests WinMD definitions and knows how to project those into the WinRT calling convention. Maybe it makes sense to just have ABIs refer to the low-level calling convention, and have a different name for the IDL + ABI system? But given how tightly these are linked, I'm not sure it's worth distinguishing between the two? Or whether talking about it in terms of like: "ABI encoding" and "ABI description" are enough? If anyone knows of better terminology for this, I'd love to hear it!</p> Why ABIs require a type system (added 2023-11-18)</h2> Jonathan Pallant</a> asked some really good questions on Mastodon about whether high-level IDLs such as WIT can conceptually be separated from the underlying encoding. From their perspective as an embedded systems engineer, ABIs are about object encodings + calling conventions. Since from their perspective it was primarily about encodings, the need for a higher-level IDL or a type system didn't really make sense. Which is a really fair perspective, and it made me scratch my head a little. But I think I've found an explanation for why the two can't be separated.</p> Take for example C. There is the programming language "C", and then there are separate calling conventions. For example on POSIX x86-64 platforms it uses the "x86-64 System V" calling convention. On 32-bit POSIX it uses the 32-bit "System V" calling convention. Gankra has written a great post about the C language and the importance of ABIs</a>.</p> I believe that when we talk about "C ABIs" (plural as there is no canonical ABI), it's not enough to just point at the encoding and calling conventions. As Gankra covered in her post; it's also important that there is a shared understanding of types. You can't implement the "C ABI" if you don't also encode what an int</code> in C is. If intmax_t</code> was not part of the "C ABI", then it would not be an issue to change it either.</p> Taking this back to Wasm Components: while there is something called the "Canonical ABI"</a> which defines the encoding of Wasm Components. If anything wants to implement the "Wasm Component ABI", they can't implement the encodings provided by that. WASI defines a number of built-ins which have a stable encoding, and thus can be considered part of the ABI too.</p> This to me feels like the most convincing argument for why when we talk about ABIs we cannot meaningfully separate the higher-level language (C or WIT) from the lower-level encoding ("x86-64 System V" or "WASI Canonical ABI"). The types defined in the higher-level language are a part of the overall contract we call "ABI" 6</a></sup>. And those types cannot be defined without also defining a type system to define those types in.</p> ^6</sup>The abi-cafe project</a> is a good example here. It tests the compatibility of various projects which output C ABI. And one of its trophy cases is an incompatibility between x86 linux clang, and gcc on how __int128</code> should be encoded when passed on-stack. And that can only be an issue if __int128</code> is considered part of the C ABI.</p> </div> Iterator as an Alias Wed, 08 Nov 2023 00:00:00 +0000 This is another short post covering another short idea: I want to explain the mechanics required to make Iterator</code> an alias for the Coroutine</code> trait. Making that an alias is something Eric Holk brought up yesterday. We then talked it through and mutually decided it probably wasn't practical. But I thought about it some more today, and I might have just figured out a way we could make work? In this post I want to briefly sketch what that could look like.</p> ⚠️ DISCLAIMER: I'm making up a ridiculous amount of syntax in this post. None of this is meant to be interpreted as a concrete proposal. This is just me riffing on an idea; which is a very different flavor from some of my multi-month research posts. Please treat this post for what it is: a way to share potentially interesting ideas in the open. ⚠️</em></p> Iterators vs Coroutines</h2> An iterator is a type which has a next</code> method, which returns an Option</code>. Every time you call next</code> it either returns Some(T)</code> with a value, or None</code> to indicate it has no data to return. A coroutine is a variation of this same idea, but fundamentally it differs in a few ways:</p> Coroutines can take arguments to their resume function. Which allows yield</code> statements to evaluate to values.</li> Coroutines can return a different type than they yield. Iterators can only yield a type, but will always return ()</code>.</li> </ul> Just so we're able to reference it throughout this post, here are the (simplified) signatures:</p> // The iterator trait </span>trait </span>Iterator { </span> </span>type </span>Item; </span> </span>fn </span>next</span>(&</span>mut </span>self</span>) -> Option<</span>Self::</span>Item>; </span>} </span> </span>// The coroutine trait </span>trait </span>Coroutine<R> { </span> </span>type </span>Yield; </span> </span>type </span>Return; </span> </span>fn </span>resume</span>(&</span>mut </span>self</span>, </span>arg</span>: R) -> CoroutineState<</span>Self::</span>Yield, </span>Self::</span>Return>; </span>} </span> </span>// The enum returned by the coroutine </span>enum </span>CoroutineState<Y, R> { </span> Yielded(Y), </span> Complete(R), </span>} </span></code></pre> Because it's useful to show, here's roughly what (I imagine) it could look like to implement both iterators and coroutines using generator function notation:</p> // Returning an iterator </span>// (syntax example, entirely made up, not an actual proposal) </span>gen </span>fn </span>eat_food</span>(</span>cat</span>: &</span>mut</span> Cat) -> yields Purr { </span>// ^ ^ </span>// \| \| </span>// \| The type yielded (output) </span>// \| </span>// The return type is always `()` </span>// </span> </span>for </span>0</span>..</span>10 </span>{ </span> </span>// yield a `Purr` ten times </span> </span>yield</span> Purr::new(); </span> } </span>} </span> </span>// Returning a co-routine </span>// (syntax example; entirely made up, not an actual proposal) </span>gen</span>(Food) </span>fn </span>eat_food</span>(</span>cat</span>: &</span>mut</span> Cat) -> Nap yields Purr { </span>// ^ ^ ^ </span>// \| \| \| </span>// \| \| The type yielded (output) </span>// \| \| </span>// \| The return type (output) </span>// \| </span>// The type `yield` resumes with (input) </span> </span>for </span>0</span>..</span>10 </span>{ </span> </span>// Every time we yield a `Purr`, yield resumes with `Food`. </span> </span>let</span> food = </span>yield</span> Purr::new(); </span> cat.</span>eat</span>(food); </span> } </span> Nap::new() </span>// We end the function by returning a `Nap` </span>} </span></code></pre> There are probably better ways of writing this. For example: imo the types passed to yield</code> should be a special entry in the arguments list. We can possibly drop the -></code> requirement for iterator-returning generator functions. And using gen</code> as a prefix probably isn't necessary either. But that's all stuff we can figure out later. I at least wanted to make sure folks had had an opportunity to see what functionality coroutines provide when writing generator functions.</p> Iterators as an alias for Coroutines</h2> Practically speaking this means that co-routines are a superset of functionality of iterators. Enough so that if we squint we might think see it work out. We can recreate the iterator trait if we implement a coroutine with this signature:</p> impl </span>Coroutine<()> </span>for </span>MyIterator { </span> </span>type </span>Yield = SomeType; </span> </span>type </span>Return = (); </span> </span>fn </span>resume</span>(&</span>mut </span>self</span>, </span>arg</span>: ()) -> CoroutineState<</span>Self::</span>Yield, ()> { .. } </span>} </span></code></pre> It's tempting to think we could create a trait alias for this. But we run into trouble here: the coroutine trait wants to be able to return a CoroutineState</code>. While Iterator</code> is hard-coded to return Option</code>. And we can't change the existing Iterator</code> trait to take anything but an option. So what should we do here.</p> Fallible functions</h2> We can't change Iterator</code>, but we can</em> change Coroutine</code>. What if rather than hard-coding that it returned a CoroutineState</code>, we instead said that it could return anything which implements the Try</code> trait? Try</code> is implemented for both not only Option</code>, but also CoroutineState</code>! And if we had try</code> or throws</code> functions, we could make that configurable like so:</p> // The coroutine trait </span>trait </span>Coroutine<R> { </span> </span>type </span>Yield; </span> try </span>fn </span>resume</span>(&</span>mut </span>self</span>, </span>arg</span>: R) -> </span>Self::</span>Yield throws { .. } </span>} </span></code></pre> edit(2023-12-01)</strong>: Yosh from the future here. Scott McMurray helpfully provided this desugaring of what an un-handwaved desugaring of this signature could look like. Crucially it shows that we could have a fully generic throws</code> clause without defining what exactly it throws:</p> // The coroutine trait </span>trait </span>Coroutine<R> { </span> </span>type </span>Yield; </span> </span>fn </span>resume</span>(&</span>mut </span>self</span>, </span>arg</span>: R) -> <</span>Self::</span>Residual as ops::Residual<</span>Self::</span>Yield>>::TryType { .. } </span>} </span></code></pre> I'm super hand-waving things away here</del>. But roughly what I'm meaning to describe here is that this can return any type implementing Try</code> which returns Self::Yield</code>, without describing the specific Try::Residual</code> type. Meaning that could equally be an Option</code>, a CoroutineState</code>, or a Result</code>. The next function should not distinguish between those. If we take that path, then we could use that to rewrite the Iterator</code> trait as an alias for Coroutine</code>. We don't have a syntax for this, so I'll just make up a syntax based on return type notation</a>:</p> // A trait alias, hard-coding both the return type and resume argument to `()`. </span>trait </span>Iterator = Coroutine<(), next(..) -> Self::Yield throws None>; </span></code></pre> There's probably a better way of writing this. But syntax is something we can figure out. Semantically this would make the two equivalent, and it should be possible to write a way to make these work the same way.</p> Also on a quick syntactical note: the throws None</code> syntax here would operate from the assumption that the right syntax to declare which type of residual you're targeting is by writing a refinement. So to use Option<T></code> as the Try</code> type, you'd write throws None</code>. For Result<T, E></code> you'd write throws Err(E)</code>. And so on. That way if you strip the throws Ty</code> notation, you'd be left with the base function signature. In a fun mirrored sort of way, I don't think we'd need to prefix fallible functions with a try fn</code> notation. Just writing throws</code> in the return type should be enough. I think the same should be true for generator functions too; just writing yields T</code> in the signature should be enough.</p> Support for pin</h2> Now, we've gone through an entire post to show how we could turn Iterator</code> into an alias for Coroutine</code>. But there is one more detail we've skipped over: in the compiler the Coroutine</code> trait is used to implement future state machines with, created using the async</code> keyword. In order for borrows to work across .await</code> points in async functions, the Coroutine</code> trait needs to support self-references. Today that's done using the Pin</code> self-type, which means the actual signature of Coroutine</code> today is:</p> trait </span>Coroutine<R> { </span> </span>type </span>Yield; </span> </span>type </span>Return; </span> </span>fn </span>resume</span>(</span>self</span>: Pin<&</span>mut Self</span>>, </span>arg</span>: R) -> CoroutineState<</span>Self::</span>Yield, </span>Self::</span>Return>; </span> </span>// ^ this changed </span>} </span></code></pre> So Coroutines</code> must be able to express self-references. And iterators currently don't support that. Which means we need to find a way to bridge the two. Luckily this is something we might be able to resolve if we consider pinning as an orthogonal capability which we can express as a trait transformer</a>. Ideally what we'd do is make Coroutine</code> and by extension Iterator</code> generic over: "may be self-referential". I talk about this more in my latest post on the async trait</a>; specifically pointing out that async iterators don't need to be self-referential - and generator functions probably do want to be able to have self-references. Making Iterator</code> an alias for Coroutine</code> further reinforces that requirement.</p> Conclusion</h2> This was a quick post to illustrate a way we could make Iterator</code> a more narrowly scoped alias for Coroutine</code>. I'm not sure we should do this; from my perspective we have a few options:</p> Never stabilize the co-routine trait. This is probably fine, since being able to pass arguments to yield</code> or change the return type from the yield type don't seem like major features. Useful, but not critical</em>.</li> Implement Coroutine</code> as a superset of Iterator</code>. Keeping both, but using blanket impls</a> make both work with one another. It's feasible, but it feels like we're creating "V2" interfaces which aren't interchangeable. You could pass an iterator everywhere a generator was accepted. But you couldn't pass a generator everywhere an iterator was accepted - unless we carve out a special exception just for this, which seems like a pretty bad hack.</li> Make Iterator</code> an alias for Coroutine</code>. That's what this post is about. It would be the way we bring some of the core unstable functionality out of the stdlib and make it available to users. Without creating an "iterator v2" trait. If we're serious about ever wanting to stabilize Coroutines, this seems preferable.</li> </ol> A nice side-effect of this approach is that we could use this as a way to normalize our syntax as well. Right now we're discussing adding "generator functions" in the 2024 edition. But we're making a distinction between "iterators", "generators", and "co-routines". If we were to adopt the aliasing model, then the iterator trait would be the same as the generator trait. And we would no longer have a need for another, separate concept for "co-routines".</p> I for one like the idea that a "generator function" returns an impl Generator</code>. And even if I don't think that working on that should be a priority, I like that we might now have a mechanism to make this a reality by creating a trait alias, hard-coding certain params, and relying on the Try</code> trait.</p> What is a team? Wed, 08 Nov 2023 00:00:00 +0000 update (2023-11-08): I've swapped the usage of "horizontal" and "vertical" in this post based on feedback.</em></p> This is going to be an ultra-short post to share an idea I've been mulling over in my head for a while. Yesterday I raised the question again about WG Async getting checkboxes for approval on decisions. For the past five or so years we've had a version of the async working group driving the design of async Rust. With the past three years being a lot more organized, with a steady roster, leadership, and mandate. WG Async getting the ability to set checkboxes for approval would mostly just be a formalization of the existing role we play in the project.</p> However, in order to give checkboxes to the Async WG there is a question about the way it should integrate into the project. Historically we've distinguished between: top-level "teams", and domain-specific "working groups". Teams are formal, first-class entities with voting rights. While working-groups are not. This current structure still exists in the project, and so in order for WG Async to get checkboxes we'd need to become a "team". And with that there is also a question about whether we should be a sub-team (like T-Types) or a top-level team.</p> Top-level teams get a seat on the council. Sub-teams are only represented on the council by their parent teams. In the case of T-Types the parent teams are "Lang" and "Compiler". In the case of WG Async the parent teams should be at least "Lang" and "Libs-API"; but arguably also "Compiler" since we tend to implement the features we've designed.</p> And this is where I think the current model runs into trouble. Ideally we'd think of the organization hierarchy as a tree: there are teams and there are sub-teams. But because sub-teams can have multiple parents, our tree is really a DAG</a>. And that becomes pretty complex to reason about. So instead I want to propose an alternate model to our organizational DAG: I think we might be better off reasoning in terms of vertical</em> and horizontal</em> teams.</p> vertical teams</strong>: These are teams such as Lang, Libs, Compiler, and Cargo. They are their a domain in their own right, and largely focus on one area of work.</li> horizontal teams</strong>: These are teams such as Types, Async, and possibly also Moderation. These are teams whose domain spans across multiple vertical teams, but rather than going broad - they specialize in a specific subject.</li> </ul> Categorizing teams like this doesn't directly help answer questions about who should have a seat on the council, nor things like membership. Those are questions of policy, and I'm intentionally keeping that out of scope for this post. But I hope it can help us explain why teams like "Async" feel different from teams like "Compiler".</p> Specifically I also believe that horizontal teams can help fill the gaps left between verticals. The Types Team exists because we identified that it's not enough to think about the language, or the compiler - but there is a need to also consider how the two interact. Similarly: WG Async doesn't just consider the async language, library, or compiler aspects. It needs to consider how the language features will be implemented, and how that in turn will affect libraries.</p> I don't know what we want to name these groups. In my opinion both vertical and horizontal teams should be teams in their own right; with neither being "more real" than the other. But as I've said: if we were to adopt that we'd need to reason about a bunch more things, because the current project structure attaches certain things to "top-level" teams right now.</p> There is also a final question about the difference between "teams" and "initiatives". The way I'm reasoning about those is as follows:</p> teams</strong>: Permanent in nature. Exist for an unbounded amount of time</em>, cover a broad area, and have a relatively steady membership.</li> initiatives</strong>: Ephemeral in nature. Exist for a bounded amount of time</em>, created for a specific purpose under a team, and spun down once that purpose has been served.</li> </ul> From my perspective the purpose of an initiative is to organize specific sets of work under a team, without requiring the people doing that work are members of that team. For example, neither Oli nor I are members of the Lang team, but the Effect Generics are a language-level effort. The Effects Initiative allows us to do this work under the purview of the Lang Team without either of us being on</em> the Lang Team.</p> And that's about it. I think if we're going to be talking about teams and sub-teams, I think it's clearer to reason about teams in terms of "vertical concerns" (fixed domains like "compiler", "language", "stdlib") and "horizontal concerns" (cross-cutting domains like: "the type system", "async support", "moderation").</p> Async Iteration III: The Async Iterator Trait Tue, 26 Sep 2023 00:00:00 +0000 This post is part of the Async Iteration series:</em></p> Async Iteration I: Async Iteration Semantics</a></li> Async Iteration II: The Async Iterator Crate</a></li> Async Iteration III: The Async Iterator Trait</a> (this post)</li> </ol> Async Functions in Traits (AFIT) are in the process of being stabilized</a> and I figured it would be a good time to look more closely at the properties they provide. In this post I want to compare AFIT-based traits with poll-based traits, using the "async iterator" trait as the driving example. But most everything which applies to async iterator, will also apply to other traits such as async read and async write.</p> In this post I will make the case that the best direction for the stdlib is to base its async traits on AFITs. The intended audience for this post is primarily my fellow members of WG-Async, as well as members of T-Lang and T-Libs. To read a summary of the findings jump ahead to the conclusion</a>. This post assumes readers are familiar with the inner workings of Rust's async systems, as well as a familiarity of the tradeoffs being discussed.</p> fn poll_ vs async fn</h2> To provide some flavor to what I'm talking about, in this post we'll be discussing the "async iterator" trait, asking the question whether we should base it on fn poll_next</code> or async fn next</code>. Here are both variants side-by-side:</p> // Using `fn poll_next`. </span>trait </span>AsyncIterator { </span> </span>type </span>Item; </span> </span>fn </span>poll_next</span>(</span>self</span>: Pin<&</span>mut Self</span>>, </span>cx</span>: &</span>mut </span>Context<'_>) -> Poll<Option<</span>Self::</span>Item>>; </span>} </span></code></pre> // Using `async fn next`. </span>trait </span>AsyncIterator { </span> </span>type </span>Item; </span> async </span>fn </span>next</span>(&</span>mut </span>self</span>) -> Option<</span>Self::</span>Item>; </span>} </span></code></pre> I expect pretty much everyone will agree that on a first look the async fn next</code>-based trait seems easier to use. Rather than needing to think about what Pin</code> is, or how Poll</code> works, we can just write our async functions the way we usually do, and it will just work</em>. Pretty neat!</p> But that's just on the surface. Does that still hold if we look more closely? Concerns have been raised about the performance of async fn next</code>, claiming not only would it perform less well. It's also alleged that async fn next</code> does not provide essential features, even going so far to claim that fn poll_next</code> is fundamentally lower-level and thus the only reasonable choice for a systems programming language. In the remainder of this post we'll be going over those claims, and show why upon closer examination they do not appear to hold.</p> Performance</h2> Let's start with the most obvious one: performance. At its core Rust is a systems programming language, and in order to properly cater to its niche it tends to only provide abstractions which have comparable performance to their hand-rolled versions. The claim is that poll_next</code> should provide better performance than async fn next</code> since we're compiling it by hand. But when actually measured, the two approaches appear to compile to identical assembly in various configurations - meaning they will have identical performance.</p> But don't just take my word for it, we can use examples to substantiate this. Let's create a simple "once" future which holds some data, and when polled it will return that data. Rather than using complex async/.await machinery, we'll be creating a new function poll_once</code> which constructs a dummy waker in-line and can be used to poll a future exactly once:</p> pub fn </span>call_once</span>() -> Poll<Option<</span>usize</span>>> { </span> </span>let mut</span> iter = </span>once</span>(</span>12</span>usize</span>); </span> </span>poll_once</span>(iter.</span>next</span>()) </span>} </span></code></pre> Let's start by evaluating what this looks like when implemented using fn poll_next</code>. We could write this as follows:</p> struct </span>Once<T>(Option<T>); </span>impl</span><T> AsyncIterator </span>for </span>Once<T> { </span> </span>type </span>Item = T; </span> </span>fn </span>poll_next</span>(</span>self</span>: Pin<&</span>mut Self</span>>, </span>_cx</span>: &</span>mut </span>Context<'_>) -> Poll<Option<</span>Self::</span>Item>> { </span> </span>// SAFETY: we're projecting into an unpinned field </span> </span>let</span> this = </span>unsafe </span>{ Pin::into_inner_unchecked(</span>self</span>) }; </span> Poll::Ready((&</span>mut</span> this.value).</span>take</span>()) </span> } </span>} </span></code></pre> When polled we project Self</code> into its fields, which is just the Option</code> type. We then call .take</code> to extract the value, or panic if there is none. This should be fairly straight forward. If poll_once</code> creates a non-atomic dummy waker (this is just the first example), the compiler will compile this code down to the following x86 assembly (compiler explorer</a>):</p> example::call_once: </span> </span>mov </span>eax</span>, </span>1 </span> </span>mov </span>edx</span>, </span>12 </span> </span>ret </span></code></pre> This assembly basically means: "Hey I've got the constant '12' over here - please move it into the return registry and then exit the function"</em>. That's about the smallest this function can be without being inlined. Now let's see what happens if we implement this code using async fn next</code>. Instead of fn poll_next</code> we can use an async function directly:</p> pub struct </span>Once<T>(Option<T>); </span>impl</span><T> AsyncIterator </span>for </span>Once<T> { </span> </span>type </span>Item = T; </span> async </span>fn </span>next</span>(&</span>mut </span>self</span>) -> Option<T> { </span> </span>self</span>.</span>0.</span>take</span>() </span> } </span>} </span></code></pre> It's nice we don't have to perform pin projections anymore (more on that later). But what's the performance like? Well, if this was slower we'd expect it to generate more assembly. So let's take a look (compiler explorer</a>):</p> example::call_once: </span> </span>mov </span>eax</span>, </span>1 </span> </span>mov </span>edx</span>, </span>12 </span> </span>ret </span></code></pre> The assembly is identical! Why is that? Well, for one: the Rust compiler is pretty good at generating fast code. But we're also in a bit of a simplified environment. So far in our examples we've not using "real" thread-safe wakers, instead basing our wakers on Rc</code>. What happens if we switch to Arc</code>-based wakers? Here's the link to a compiler explorer</a> comparing the two. It now generates a lot more assembly than before (yay atomics), but luckily we can use diff(1)</code> to compare the output:</p> example::call_once: </span> ; 21 lines of assembly + calls to another 118 lines </span></code></pre> yosh@MacBook-Pro</span> scratch % pbpaste > one.rs </span>yosh@MacBook-Pro</span> scratch % pbpaste > two.rs </span>yosh@MacBook-Pro</span> scratch % diff one.rs two.rs </span>yosh@MacBook-Pro</span> scratch % </span></code></pre> The diff</code> output is empty, meaning there are no differences even if we Arc</code>s to correctly construct our wakers, it just generates a lot more code. But okay fine, maybe there are more differences? After all: fn poll_next</code> has access to the Waker</code> and can return Poll</code>, meaning it has low-level control over the future state machine while async fn next</code> does not. What happens if we want to provide low-level control over the future state machine from async fn next</code>?</p> Luckily we've stabilized a simple mechanism for this already: std::future::poll_fn</code></a>. This function provides the ability to access the low-level internals of any</em> future, including AFITs. Let's lower our example to make use of this, shall we?</p> pub struct </span>Once<T>(Option<T>); </span>impl</span><T> AsyncIterator </span>for </span>Once<T> { </span> </span>type </span>Item = T; </span> async </span>fn </span>next</span>(&</span>mut </span>self</span>) -> Option<T> { </span> future::poll_fn(\|</span>_cx</span>\| </span>/ -> Poll<Option<T>> / </span>{ </span> </span>// We have access to `cx` here which contains the `Waker`. </span> Poll::Ready((&</span>mut </span>self</span>.value).</span>take</span>()) </span> }).await </span> } </span>} </span></code></pre> This seems simple enough: whenever we want to do anything low-level inside of an async fn</code>, we can use poll_fn</code> to drop into the future state machine. This should work not just for the async version of the iterator trait, but for all</em> async traits. There is more to be said about how this interacts with pinning and self-referential types, but we'll cover that in more detail later on in the post. To close this out though: what does this compile to if we call it using "real" wakers? (compiler explorer</a>):</p> example::call_once: </span> ; 21 lines of assembly + calls to another 118 lines </span></code></pre> yosh@MacBook-Pro</span> scratch % pbpaste > two.rs </span>yosh@MacBook-Pro</span> scratch % diff one.rs two.rs </span>yosh@MacBook-Pro</span> scratch % </span></code></pre> That's right: the output remains the same. This gives us a pretty good clue about what is happening here. Inside the compiler async fn next</code> is desugared to a future, just like fn poll_next</code> is. And because of basic inlining and const-folding optimizations, the resulting state machines are identical - which means that the resulting assembly is identical too. This is exactly how zero-cost abstractions are supposed to work, and is the entire premise of Rust's async system</a>. If we ever find a case where the optimizer doesn't perform those basic optimizations we can then treat that as a bug in the compiler - not a limitation of the design.</p> Self-Referential Iterators</h2> When people say that "async iterator is not the async version of iterator"</em> they are correct. Well, sort of. If we look at existing</em> implementations</a> that is true: it doesn't quite work like the async version of iterator. Instead what it really is is the async version of "pinned iterator". Which is not a trait we currently have, but there certainly is a case to be made for it. Instead it's better to ask whether async iterator should</em> be the "async version of iterator" - and I certainly believe it should be 1</a></sup>.</p> ^1</sup>Incidentally that has also been the framing of the trait WG-async has been communicating to T-lang and T-libs</a>, who have signed off on it. I'm not suggesting that this decision should bind</em> us (I don't like to rules lawyer). What I'm instead trying to show with this is that this has been an accepted framing of what the design should achieve for years now, and we've already rejected the framing that "async iterator" (or "stream") should be its own special thing. That certainly can be changed again, but it is not a novel insight by any stretch.</p> </div> Let me explain what I mean by this using examples. In Rust the base iterator API has an associated type Item</code>, a function next</code> which takes a mutable reference to self</code>, and returns an Option<Self::Item></code>:</p> trait </span>Iterator { </span> </span>type </span>Item; </span> </span>fn </span>next</span>(&</span>mut </span>self</span>) -> Option<</span>Self::</span>Item>; </span>} </span></code></pre> If we did a direct translation to async Rust, we'd have an API which instead of exposing an fn next</code> exposed an async fn next</code>. The only real difference here is the addition of the async</code> keyword:</p> trait </span>AsyncIterator { </span> </span>type </span>Item; </span> async </span>fn </span>next</span>(&</span>mut </span>self</span>) -> Option<</span>Self::</span>Item>; </span>} </span></code></pre> However, when we look at the ecosystem Stream</code> trait</a>, or the currently unstable AsyncIterator</code> API</a> they are not implemented in terms of async fn next</code>. Instead they provide an fn poll_next</code> which takes both a pinned reference to self</code>, a mutable reference to the waker context, and wrap the return type in Poll</code>:</p> trait </span>AsyncIterator { </span> </span>type </span>Item; </span> </span>fn </span>poll_next</span>(</span>self</span>: Pin<&</span>mut Self</span>>, </span>cx</span>: &</span>mut </span>Context<'_>) -> Poll<Option<</span>Self::</span>Item>>; </span>} </span></code></pre> In the previous section we've already discussed how you can get access to the waker context from inside an async function by using poll_fn</code>. So we can pretty much ignore the waker context and the Poll</code> in the return type. That leaves the change in the self</code> type. Our async fn next</code> takes &mut self</code>, while this variant takes Pin<&mut Self></code>. This isn't necessary for the core functionality of async iterator, since it is pinning more than needed. So simply put, what we've just written is in fact the async version of this trait:</p> trait </span>PinnedIterator { </span> </span>type </span>Item; </span> </span>fn </span>next</span>(</span>self</span>: Pin<&</span>mut Self</span>>) -> Option<</span>Self::</span>Item>; </span>} </span></code></pre> Here we have a non-async version of iterator which takes self as a pinned reference. This is useful if you ever need to write an iterator which can operate on self-referential structs. For example: if we ever start thinking of stabilizing generator functions, we want them to be able to hold references across yield</code> points. That will require self-referential iterators.</p> An important insight of this is that the question of whether iterator should be pinned is orthogonal to whether it is async. Which is illustrated by the fact that we can reformulate a "pinned asynchronous iterator" just fine using AFITs:</p> trait </span>PinnedAsyncIterator { </span> </span>type </span>Item; </span> async </span>fn </span>next</span>(</span>self</span>: Pin<&</span>mut Self</span>>) -> Option<</span>Self::</span>Item>; </span>} </span></code></pre> This can be combined with the poll_fn</code> function as we showed in the previous section to recreate the low-level semantics of fn poll_next</code>, providing access to both a pinned self-type and the future's waker argument. To put it plainly: "is async" and "is pinned" are orthogonal features, and fn poll_next</code> needlessly combines both</strong>.</p> Unpin Bounds</h2> People occasionally ask me about the Unpin</code> trait when I talk about async versions of traits. For example if you compare Iterator::next</code></a> and futures::stream::StreamExt::next</code></a>, you will see that the latter has an extra where Self: Unpin</code> bound.</p> // `Iterator::next` </span>fn </span>next</span>(&</span>mut </span>self</span>) -> Option<</span>Self::</span>Item>; </span> </span>// `FuturesExt::next` </span>fn </span>next</span>(&</span>mut </span>self</span>) -> Next<'_, </span>Self</span>> </span>where </span> </span>Self</span>: Unpin; </span>// This is different </span></code></pre> This extra Unpin</code> bound is only needed when a trait is implemented in terms of poll functions - which by design take Pin<&mut Self></code>. And so we need a way to later on opt out of those bounds. You can see this same mechanism in action with the other poll-based traits, such as AsyncWriteExt::write</code></a> which also has a Self: Unpin</code> bound.</p> Instead if we recognize that the async counterparts to Rust's core traits don't actually need to be pinned in order to be implemented, we can drop Pin<&mut Self></code> from the signature. And since our type isn't pinned to begin with, we no longer have to opt-out of it being pinned via Unpin</code> meaning all the extra Unpin</code> bounds go away. You can see this in action in the async-iterator</code> crate</a> which provides a diverse range of methods on async iterator, none of which require additional Unpin</code> bounds to function.</p> Implementation vs Usage</h2> To stay on the topic of API-shapes: one major downside of fn poll_next</code> is that the "method to implement" and "method to call" are different methods. In the regular iterator trait, there is only one method next</code> which is both implemented and called. Instead the poll-API is only meant to be implemented</em>, and in virtually all cases the next</code> function is the one you want to call. This is a major deviation of how all other traits work in the stdlib today.</p> This isn't just limited to async Iterator</code> either. Presumably we'd want to adapt this approach for all</em> traits in the stdlib. That means users of async Rust would need to think of traits in the stdlib as somehow "different", and remember that they cannot directly implement the methods they're calling. Among others, the following APIs would be affected:</p> poll-based stdlib traits</strong></p> trait name</th> to be implemented</th> to be called</th> is same?</th></tr></thead> async Read</strong></td> fn poll_read</code></td> async fn read</code></td> ❌</td></tr> async Write</strong></td> fn poll_write</code></td> async fn write</code></td> ❌</td></tr> async BufWrite</strong></td> fn poll_fill_buf</code></td> async fn fill_buf</code></td> ❌</td></tr> async Seek</strong></td> fn poll_seek</code></td> async fn seek</code></td> ❌</td></tr> </tbody></table> Instead, if we base these traits on async fn</code>, the method to implement and the method to call are identical. And as we've covered earlier, if anyone would want to manually author a poll</code>-based state machine for any of these traits, poll_fn</code> provides a uniform way to do so for all</em> async traits:</p> async-fn based stdlib traits</strong></p> trait name</th> to be implemented</th> to be called</th> is same?</th></tr></thead> async Read</strong></td> async fn read</code></td> async fn read</code></td> ✅</td></tr> async Write</strong></td> async fn write</code></td> async fn write</code></td> ✅</td></tr> async BufWrite</strong></td> async fn fill_buf</code></td> async fn fill_buf</code></td> ✅</td></tr> async Seek</strong></td> async fn seek</code></td> async fn seek</code></td> ✅</td></tr> </tbody></table> This might seem like a minor point, but we have to consider that every deviation from existing norms is a point of friction for users. To zoom out slightly: I don't believe that async Rust inherently needs to be much more difficult than regular Rust. But the missing language features, combined with subtle differences like these, eventually add up and create an experience which is sufficiently different that the resulting system feels like an entirely different language. When in reality it does not need to be. For good measure here are the existing non-async stdlib traits:</p> non-async stdlib traits</strong></p> trait name</th> to be implemented</th> to be called</th> is same?</th></tr></thead> Read</strong></td> fn read</code></td> fn read</code></td> ✅</td></tr> Write</strong></td> fn write</code></td> fn write</code></td> ✅</td></tr> BufWrite</strong></td> fn fill_buf</code></td> fn fill_buf</code></td> ✅</td></tr> Seek</strong></td> fn seek</code></td> fn seek</code></td> ✅</td></tr> </tbody></table> Object Safety</h2> So far we've only discussed the implementation side of the traits. However that isn't the complete story, and we need to consider auto traits and other subtle semantics too. So let's start looking at those, starting with object-safety</a>. Out of the box poll</code>-based traits are dyn-safe. Say we wanted to implement a poll-based version of async iterator which can produce an infinite number of meows, we could create a dyn variant like so (playground</a>):</p> struct </span>Cat; </span>impl </span>AsyncIterator </span>for </span>Cat { </span> </span>type </span>Item = String; </span> </span>fn </span>poll_next</span>(</span>self</span>: Pin<&</span>mut Self</span>>, </span>_cx</span>: &</span>mut </span>Context<'_>) -> Poll<Option<</span>Self::</span>Item>> { </span> Poll::Ready(Some("</span>meow</span>".</span>to_string</span>())) </span> } </span>} </span> </span>fn </span>dyn_iter</span>() -> Box<dyn AsyncIterator<Item = String>> { </span> Box::new(Cat {}) </span>} </span></code></pre> This is the same as any other dyn-safe trait, and doesn't requiring any additional steps. Nice! Now what happens if we try and rewrite it to use async fn next</code>. Well, we would probably try and write it like so (playground</a>):</p> #![</span>feature</span>(async_fn_in_trait)] </span> </span>struct </span>Cat; </span>impl </span>AsyncIterator </span>for </span>Cat { </span> </span>type </span>Item = String; </span> async </span>fn </span>next</span>(&</span>mut </span>self</span>) -> Option<</span>Self::</span>Item> { </span> Some("</span>meow</span>".</span>to_string</span>()) </span> } </span>} </span> </span>fn </span>dyn_iter</span>() -> Box<dyn AsyncIterator<Item = String>> { </span> Box::new(Cat {}) </span>} </span></code></pre> However if we now try and compile this code we now get the following error:</p> error[E0038]: the trait `AsyncIterator` cannot be made into an object </span> --> src/lib.rs:16:22 </span> \| </span>16 \| fn dyn_iter() -> Box<dyn AsyncIterator<Item = String>> { </span> \| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ `AsyncIterator` cannot be made into an object </span> \| </span>note: for a trait to be "object safe" it needs to allow building a vtable to allow the call to be resolvable dynamically; for more information visit <https://doc.rust-lang.org/reference/items/traits.html#object-safety> </span> --> src/lib.rs:5:14 </span> \| </span>3 \| trait AsyncIterator { </span> \| ------------- this trait cannot be made into an object... </span>4 \| type Item; </span>5 \| async fn next(&mut self) -> Option<Self::Item>; </span> \| ^^^^ ...because method `next` is `async` </span> = help: consider moving `next` to another trait </span></code></pre> This would not happen if we were using the async-trait crate</a>, it only happens if we use the AFIT language feature. But what exactly is going on here? In order for async traits to work in dyn contexts, we need to find a place to store them first. In the async-trait</code> crate this is always a Box</code>, but in stable Rust we can't just do that because we've pinky-promised not to perform any "hidden" allocations 2</a></sup>. Solving this is pretty complicated, and will require a feature like dyn</code></a> to land. With dyn</code> rather than calling Box::new</code> you'd need to call a "dyn adapter" instead. For example</a>:</p> ^{2</sup> Though in the past we have</em> used allocations in language features as placeholders: that's how async/.await</code> was originally stabilized in 2019. It wasn't until a year or so later that support landed for async functions which didn't box in their desugaring.</p> </div>}fn </span>dyn_iter</span>() -> Box<dyn AsyncIterator<Item = String>> { </span> Boxed::new(Cat {}) </span>// NOTE: using the `Boxed` dyn-adapter, not `Box`. </span>} </span></code></pre> Needing to replace Box::new</code> with Boxed::new</code> is not a major difference. And it's still unclear what the upper bound on the ergonomics of dyn async traits are, since work on it has been paused for the past year in favor of stabilizing AFIT first. But it's going to be pretty confusing if certain async traits require a Box</code>, but other async traits require a Boxed</code> notation.</p> I believe the better direction is for all async traits in the stdlib to use the same mechanism to work with dyn, even if initially it's slightly less convenient at first. We can then gradually work on improving those ergonomics for all async traits, both in the stdlib and ecosystem. This ensures consistency and enables us to design solutions which are shared by the entire async ecosystem and preventing a possible permanent bifurcation of the async trait space.</p> "Cancellation Safety"</h2> Another subtle outcome here is the interaction between async traits and the property has been historically called: "cancellation safety". I'm putting it in quotes because the property is not actually about whether it is "safe to cancel" a future. A few months ago I gave a talk</a> about this, which I'll summarize in this section. I'll explain what "cancellation safety" is, how it currently falls short, and how may be able to fix those shortcomings. I particularly want to show how we can bring the "cancellation safety" property into the type system, which could enable async functions (including AFIT) to automatically provide it.</p> "Cancellation safety" refers to the ability to drop and recreate futures without any "side-effects" being observable. It's important to note that the word "safety" in this context has nothing to do with memory safety: in the absence of linear types</a>, all types in Rust must be memory-safe to drop, including futures. The "safety" in "cancellation safety" refers to logical correctness only. Being able to drop and recreate futures is the most relevant to the select! macro</a> which commonly does that in a loop. But it can also come in useful if you're ever authoring future state machines by hand.</p> /// # Cancel safety </span>/// </span>/// This method is cancel safe. If you use it as the event in a </span>/// </span>`tokio::select!`</span> statement and some other branch completes </span>/// first, then it is guaranteed that no data was read. </span></code></pre> As I've mentioned, "cancellation safety" as a property currently only exists in documentation. This means that in order to learn whether you can drop and recreate futures without any actions you need to consult the documentation for the method. The difference between async fn next</code> and fn poll_next</code> is that for the former the property will be documented on the implementation, whereas for the latter the property will be documented on the trait definition. That practically means that with fn poll_next</code> you'll be able to remember that all</em> calls to next</code> are "cancellation-safe". Whereas with async fn next</code> you'll need to remember that for each implementation (or more likely: remember which impls aren't "cancellation-safe").</p> But we should be legitimately asking ourselves whether using documentation for this is the best we can do. RTFM</a> is not really how we do things in the Rust project, instead much preferring if the compiler can tell you when you've messed up, which can then explain what to do instead. This is possible because of Rust's "type safety"</a> guarantee, and APIs such as select!</code> are decidedly not type-safe right now. Which is why even after successfully compiling code, people regularly find runtime bugs in their select!</code> statements.</p> I believe a better direction would be to bring "cancellation safety" out of the docs and into the type system. We could do this by introducing a new subtrait</a>, which I'll refer to in this post as AtomicFuture</code> (but we can call it anything we like really, the semantics are what I care about most). We already have some precedent for this in the iterator family of traits, where for example the unstable TrustedLen</code></a> trait provides additional guarantees on top of the existing Iterator</code> trait. I imagine this would look something like this:</p> //! ⚠️ This uses a placeholder name and is intended as a design-sketch only. ⚠️ </span> </span>/// A future which performs at most a single async operation. </span>trait </span>AtomicFuture: Future {} </span></code></pre> One of the downsides of the current "cancellation safety" system is that we have to manually inspect anonymous futures whether they're cancellation-safe or not. I believe by bringing "cancellation safety" into the type system the compiler should be able to figure it out automatically if the following cases are met:</p> No local state:</strong> The async context must contain at most one .await</code> point. That way if a future is cancelled it will never occur between two .await</code> points.</li> Virality:</strong> All futures awaited inside the async</code> context implement the trait.</li> </ol> Any async fn</code> or async {}</code> block would automatically be able to implement AtomicFuture</code> if those requirements are upheld, which I believe is something which shouldn't be too hard to figure out from the generated state machine 3</a></sup>. Maybe there is a case to be made for syntax for this too; I'm not sure. But that's something we can figure out later.</p> ^{3</sup> Also: this is something we'll want to do regardless if we ever get generator functions. It would be pretty bad if gen fn</code> couldn't automatically implement marker traits on the type it returns.</p> </div>}// ✅ -> impl Future + AtomicFuture </span>async </span>fn </span>foo</span>() -> </span>u32 </span>{ </span>12 </span>} </span> </span>// ✅ -> impl Future + AtomicFuture </span>async </span>fn </span>foo</span><F: AtomicFuture>(</span>fut</span>: F) -> </span>F::</span>{ </span> fut.await </span>} </span> </span>// ❌ -> impl Future </span>async </span>fn </span>foo</span><F: Future>(</span>fut</span>: F) -> </span>F::</span>{ </span> fut.await </span>} </span> </span>// ❌ -> impl Future </span>async </span>fn </span>foo</span><F: AtomicFuture>(</span>fut1</span>: F, </span>fut2</span>: F) { </span> fut1.await; </span> fut2.await; </span>} </span></code></pre> The compiler will only be able to automatically figure out whether AtomicFuture</code> can be implemented for futures returned by async fn</code> and async {}</code>. For manually implemented futures the author of the future will need to uphold those guarantees themselves. The bar to meet there is that the future should only perform a single operation, and then return. read</code> is a good example of a "stateless future", while the future join</code> operation</a> is a good example of a future which is not.</p> As a basis I think this is pretty good, but there are still some cases we haven't covered yet. For example tokio's Mutex::lock</code> operation</a>: only performs one operation, but when it's dropped and recreated it'll be the last in the unlock queue again - which in pathological cases could lead to certain unlocks never being scheduled. It's not marked "cancellation-safe" in the tokio docs, but the reason for that is not covered by the rules we've described so far.</p> I think practically we'll want to play around with the rules a little. I don't think we want or even can practically nail down what a "side-effect" is in Rust. But if we intentionally don't make the presence of AtomicFuture</code> a safety invariant, we can probably get away by adding broad rules such as:</p> "Cancelling and recreating an AtomicFuture</code> should not cause logic errors"</p> </blockquote> What does that exactly mean? That's up for interpretation. But that might be fine for our purposes. I think it's worth experimenting a little, so we can settle on something that works for us. Ideally a definition which can be automatically implemented by the compiler for async blocks and functions, but if we can't that's probably also okay. The key takeaway here should be that if we care enough about "cancellation-safety" to make it a guarantee of our public APIs, we should be trying to bring it into the trait system instead.</p> Pin Ergonomics</h2> I think if we're to have an honest conversation about poll-based APIs, we should acknowledge that working with Pin</code> is not a great experience for most people. I'm a world-leading expert in async Rust, and four years post stabilization I still regularly struggle to use it. I've seen folks on both T-lang and T-libs struggle with it. I've seen very capable engineers at Microsoft struggle with it. And if experienced folks struggle to use Pin, I don't think we can reasonably expect those with less experience to use it without much problems either.</p> I've covered some of the unique difficulties of Pin</code> on this blog before. I believe pin is hard to use in part because pin inverts Rust's usual semantics via a combination of auto-traits and wrapper structs. Another reason is because it relies heavily on the concept of "pin-projection" which requires unsafe</code> and is not directly represented in either the language or the stdlib. Which in turn means that even the most common interactions with Rust's pinning system rely third-party ecosystem crates such as pin-project</a> and until recently pin-utils</a>.</p> We currently don't have a clear path to fix Pin's ergonomics. If we want Rust to provide a way to make pin projections safe, we'll at least need to make a breaking change to the Unpin</code> trait 4</a></sup>. As well as a whole lot of design work</a> to integrate it into the language. And while this may be something we'll want to do eventually, we really need to ask ourselves whether this is something we want to do now</em>. Because taking on one project necessarily means not taking on another.</p> ^{4</sup> Safe pin projections in the language necessarily require that Unpin</code> is an unsafe</code> trait. It's currently not unsafe, and changing it would be a breaking change.</p> </div> I honestly think async functions in traits, async drop, async iteration, and async closures all are far more important things to work on right now, and they should take priority in WG-async over directly fixing Pin's rough edges. The most effective strategy to reduce the cost of Pin</code> we have in Rust right now is to simply make pin less common</strong>. If Pin</code> is only needed to implement intrusive collections, self-referential types, and manual future state machines then for the time being we think of it as an experts-only system. And once we have more bandwidth we can revisit it later to tidy it up.</p>}Evolution</h2> A thing that worries me in particular about fn poll_next</code> is its inability to co-evolve with Rust's needs. As we've seen we're not just considering adding an "async version of iterator"</a>, we're also talking about "a pinned version of iterator". But there are other flavors of iterator being discussed as well, such as: "a fallible version of iterator"</a>, "a lending version of iterator"</a>, "a const version of iterator", and so on. It's impossible to know today which requirements Rust will have ten years from now. All we really know is that the decisions we make then will be affected by the decisions we make today 5</a></sup>.</p> ^{5</sup> Ten years ago we were unsure whether it was even possible to publish a new systems programming language. Five years ago we were unsure whether Rust would be able to escape its browser niche and reach mainstream adoption. Flash forward to today, and Rust is being used in the hearts of major operating systems (Android, Linux, and Windows). As well as every packet on the internet more likely than not being routed through some Rust code at some point (Cloudflare, AWS, and Azure). I don't think we can reasonably anticipate all the requirements Rust will face ten years from now. So I believe one of the most important things for a programming language is to find ways to keep evolving the language in the face of changing requirements.</p> </div>}Say for a second we did add an fn poll_next</code>-based version of iterator. If we wanted to say, add a lending version of iterator. Then we'd probably also want to add a lending version of async iterator</a> too. That would be four separate families of traits, and that's just for three variants. If we actually wanted to add support for "pinned iterators" or "fallible iterators" we'd be looking at nine or maybe even seventeen different families of traits. And nobody reasonably wants that.</p> This is the reason why I believe async fn next</code> is the better direction. If we add it to the stdlib via an effect-generic mechanism, both "iterator" and "async iterator" could be served using a single trait which is generic over the async effect. But we could use the same mechanism for other traits such as Read</code> and Write</code>, but also maybe some less common traits like Into</code> and Display</code>.</p> //! A version of iterator which can be implemented as either sync or async. </span>//! This uses placeholder syntax just to illustrate the idea. </span> </span>#[</span>maybe_async</span>] </span>trait </span>Iterator { </span> </span>type </span>Item; </span> #[</span>maybe_async</span>] </span> </span>fn </span>next</span>(&</span>mut </span>self</span>) -> Option<</span>Self::</span>Item>; </span>} </span></code></pre> fn poll_next</code> forces us to duplicate existing interfaces in order to expose those same capabilities. While async fn next</code> enables us to extend existing interfaces with new capabilities. It doesn't immediately solve all of the limitations of iterator; effect generics won't provide an answer for how to add support for "lending" or "pinned" iterator variants. But it provides an answer to some other problems, like how to add support for async, const, and fallibility. And in doing so encourages us to find similar solutions for the problems which remain, even if we don't yet know what they'll look like.</p> Conclusion</h2> In this post I've presented a case for basing the async Iterator trait on async fn next</code> over fn poll_next</code>. To summarize the arguments:</p> Performance</strong>: Iterators based on async fn next</code> and fn poll_next</code> generate identical code, which means they have identical performance profiles. Implementations which need to access the low-level future state machine of async fn next</code> can do so using poll_fn</code></a>.</li> Self-referential iterators</strong>: fn poll_next</code> is the async version of PinnedIterator</code>, async fn next</code> is the async version of Iterator</code>. Adding a trait for pinned async iteration could be useful, but realistically it should mirror a synchronous "pinned iterator" design. And it could be written using async fn next</code> taking a pinned self-type.</li> Unpin bounds</strong>: The presence of extra Unpin</code> bounds are a way in which async traits presently meaningfully deviate from their synchronous counterparts. "async iterator" meaningfully deviates from its synchronous counterparts. It makes it seem like deeper changes are needed to get to the same async functionality. By not requiring self: Pin<&mut Self></code>, no additional where Self: Unpin</code> bounds are required as seen on methods such as StreamExt::next</code></a>.</li> Implementation vs Usage</strong>: async fn next</code> has one method to both implement and use. fn poll_next</code> requires two methods: one which must be implemented, and another which must be called. This is inherently more difficult to use, and as a mechanism is unique to poll functions other than Future</code>.</li> Object Safety</strong>: async fn next</code> and fn poll_next</code> need to use subtly different mechanisms to create dynamically dispatched objects. Adding support for dynamic dispatching is still in-progress for AFITs, but that seems like it's mostly a matter of time. Neither approach is likely going to be much harder to use than the other, but the subtle differences may be difficult to internalize for users. Diagnostics seem like they'll play an important role.</li> "Cancellation Safety"</strong>: "cancellation safety" is currently a documentation-only property. If async iterator is based on fn poll_next</code> users can blindly assume every implementation provides a cancellation-safe next</code> method. If async iterator is based on async fn next</code> users will have to check the implementation's docs to learn whether the next</code> method is "cancellation-safe". However, rather than keeping "cancellation-safety" as a documentation-only property, we should probably instead be working to bring into the type system instead.</li> Pin Ergonomics</strong>: The Pin</code> family of APIs in Rust is notorious for being difficult to use. One of the most effective ways we have to reduce the difficulty of Pin is by limiting user's exposure to it. By basing Rust's core async traits on async functions we can reduce the number of pin-based APIs, making async Rust more accessible to more people.</li> Evolution</strong>: async fn next</code> can be implemented as an extension</em> to the existing iterator trait via effect generics. fn poll_next</code> would most need to be implemented as a standalone trait, resulting in a manual duplication</em> of the APIs. Support for async is not the last feature we'll want to add to iterators: there are currently ecosystem demands to support self-referential iteration, lending iteration, and fallible iteration. It's not practical to add individual traits for all of these and their combinations. Nor is it reasonable we can anticipate all needs which may arise in the future. By extending rather than duplicating we</li> </ul> I believe basing the async Iterator trait on async fn next</code> to be superior across all axes, and I hope this post sufficiently makes that case. In a future post I'd like to round out this series by covering the desugaring of an async iteration notation. Together with an RFC for effect-generic trait definitions, this will be one of the last steps necessary step before we can re-RFC RFC 2996: Async Iterator</a> to cover the full scope of async iteration.</p> Thanks to Eric Holk for reviewing an earlier draft of this post.</em></p> nesting allocators Wed, 09 Aug 2023 00:00:00 +0000 I regularly point people to Tmandry's 2021 post "Contexts and Capabilities in Rust"</em></a>. While nobody is actively working on the design at the moment, it's more because of prioritization than anything else. It seems like it could make certain usage patterns a lot nicer in Rust, and one of those is probably working with custom allocators. I've been working a lot with slab allocators</a> recently, and I'd love it if they were easier to work with in Rust. So I wanted to take a moment to talk about allocators, capabilities as a language feature, and why I believe that would work well.</p> Kinds of allocation</h2> Memory, at least semantically, is best thought of as a contiguous sequence of bytes. Allocators manage these bytes and do the bookkeeping for who is using what. Allocators can take any type of object ("heterogenous types") and place them in memory. And once we're done with an object, the allocator will reclaim the memory. Because not all objects are the same size this can leave gaps in memory, so occasionally the allocator spends some time reorganizing its internals (think: defragmentation). In practical terms, every program wants to have a single allocator to manage the entire</em> program. This root allocator is what we refer to as the global allocator</em>.</p> But where global allocators are great</em>, they are not perfect</em>. Typically when we write programs we'll see usage patterns in our main loop. For example: if we're writing an HTTP server we will typically allocate data for the request headers and body. And once that request is handled, we de-allocate it and allocate a new request. We can use our knowledge of our usage patterns to handle allocations in a more targeted way. The way we do this is by obtaining a relatively large-ish chunk of memory from our system allocator</em>, and then using that as the heap space for our custom allocator. These custom allocators are also known as "region-based allocators"</em> or "arena allocators"</em>. Their key benefit is that their memory management strategies are very simple, which can make allocations so cheap they might as well be free.</p> There are many different types of arena allocators, but these are the ones I use the most:</p> slab allocators:</strong> For example the slab</a> crate in Rust. This is ideal for when you perform many homogenous, short-lived allocations over a longer period.</li> bump allocators:</strong> For example the bumpalo</a> crate in Rust. This is ideal for phase-oriented allocations</em>, where you allocate a number of heterogenous objects, say, on every iteration of a loop and then de-allocate them at the end of it.</li> </ul> While not every crate lends itself equally well to it, it's even possible to combine arena allocators to further customize allocation behavior.</p> Nesting Allocators</h2> The bumpalo</a> crate comes with an optional collections</code> feature</a> which can be enabled to grant access to String</code> and Vec</code> types which are identical to the stdlib's versions other than the fact that they allocate on the bump allocator. Using it looks like this:</p> use </span>bumpalo::{Bump, collections::Vec}; </span> </span>let</span> b = Bump::new(); </span>// Create a new bump allocator </span>let</span> v: Vec<</span>i32</span>> = Vec::new_in(&b); </span>// Allocate on the bump allocator </span></code></pre> This is an example of nesting allocators</em>. The system allocator hosts the bump allocator, and the bump allocator in turn hosts the vector. The main downside of this is that we're not using std::vec::Vec</code>, but have to use bumpalo's custom vec implementation. That's not ideal, not just because we now have a second Vec</code> type, but because we also don't have other collection types such as HashMap</code>, HashSet</code>, and every other type out there on crates.io.</p> Passing Allocators</h2> The plan is to solve this limitation via the nightly allocator_api</code> feature in the stdlib. This extends the stdlib with methods such as Vec::new_in</code></a> which can take a custom Allocator</code></a> argument to host the allocations in. I don't have any particular opinions about the exact Allocator</code> trait, but I do have opinions about the way this bubbles up to collection types.</p> Roughly speaking, the current design adds a new allocator generic param to collection types which defaults to the global allocator (Vec<T, A = GlobalAllocator></code>). In order to uniformly make use of this, every constructor method for every collection type needs to provide new _in</code> variants of their existing collections. At first that seems reasonable; but it quickly runs into issues.</p> The first issue is usage with macros, which we can probably overcome. Macros like vec!</code> perform allocations, and in order to work with custom allocators they should probably take an optional third param for the allocator. The second issue is a lot harder: traits can't take extra arguments. For example, if you have an &str</code>, and you want to convert it to String</code> but use a bump allocator you can't keep using the existing into</code> method. And because the into</code> trait is part of core</code>, we can't just add a new into_in</code> method to the Into</code> trait either. The only real option is probably a new trait in alloc</code>:</p> let</span> s = "</span>hello</span>".</span>to_owned</span>(); </span>// 1. Uses the global allocator </span>let</span> bump = Bump::new(); </span>let</span> s = "</span>hello</span>".</span>to_owned_in</span>(&bump); </span>// 2. Uses a bump allocator </span> </span>let</span> s: String = "</span>hello</span>".</span>into</span>(); </span>// 1. Always uses the global allocator </span> </span>// 2. ❌ Unclear how to create a variant </span> </span>// of this with a custom allocator </span></code></pre> While the number of collection types and constructor methods are limited, things can quickly spiral out of hand when we factor in composition</em>. In order for things to work uniformly across the ecosystem, every type which holds a Vec</code> internally would need a second _in</code> variant for all of their constructors and conversions. That kind of duplication is uncompelling, meaning it's likely we're only going to see some</em> types adopt support for custom allocation while the majority doesn't. Not unlike how #[no_std]</code> is not uniformly supported throughout the crates ecosystem today.</p> Scoped Allocators</h2> What we're really struggling with is that the global allocator acts, in a capability-sense, as an ambient authority</a>. Which is to say: every function in Rust has access to the global allocator without functions needing to take allocators explicitly as arguments. The allocator is just always present. Except when we're in #[no_std]</code> environments, where there is no allocator. Or when we're trying to change which allocator we're trying to take.</p> The _in</code> family of methods is an attempt to remedy the issue of having an ambient allocator. But that has some drawbacks: most notably the bifurcation of APIs. But also: it seems like that would be quite annoying to use. Manually having to pass some form of alloc</code> argument down to every function call doesn't exactly seem ideal.</p> Instead Tmandry's capabilities design</a> seems like a much more pleasant direction. It acknowledges the existence of ambient capabilities, and instead just asks that we express them at the signature level. Because they effectively function as implicits, they just carry through to all argument in scope which need them:</p> let</span> s = "</span>hello</span>".</span>to_owned</span>(); </span>// 1. Uses the global allocator </span>with alloc = Bump::new() { </span> </span>let</span> s = "</span>hello</span>".</span>to_owned_in</span>(&bump); </span>// 2. Uses a bump allocator </span>} </span> </span>let</span> s: String = "</span>hello</span>".</span>into</span>(); </span>// 1. Always uses the global allocator </span>with alloc = Bump::new() { </span> </span>let</span> s = "</span>hello</span>".into; </span>// 2. ✅ Uses a bump allocator </span>} </span></code></pre> Compositions with functions and types would no longer require manually passing allocators into arguments, but merely acknowledging the existence of an allocator in the first place. That would mean we don't have to provide new _in</code> methods which take allocators, but instead we could keep using the existing new</code> methods with some minor modifications.</p> // Vec's design on nightly today, providing both `new` and `new_in` methods </span>impl </span>Vec<T, A = Global> { </span> </span>fn </span>new</span>() -> </span>Self </span>{ ... } </span>} </span>impl </span>Vec<T, A> { </span> </span>fn </span>new_in</span>(</span>alloc</span>: A) -> </span>Self </span>{ ... } </span>} </span> </span>// A design of vec using capabilities, using just a single `new` method. </span>impl </span>Vec<T> </span>with alloc = Global, </span>{ </span> </span>fn </span>new</span>() -> </span>Self </span>{} </span>} </span></code></pre> Obviously questions remain about how to exactly model capabilities. Many types in the stdlib liberally make use of ambient capabilities (e.g. allocation, filesystem, networking, etc.) which we'd need to continue allowing to work as it does today. But there is a question about how we can instead weave through user-provided capabilities through the entire stdlib and crates.io ecosystem. The answer requires having a way to continue relying on ambient capabilities, but opt-out if you want to. Which is not entirely unlike const</code>.</p> The final benefit to this is that this would solve one of the current blockers we have for string-literals. The obvious design for that would be to allow the s</code> prefix on literals to create allocated strings. But because that would act as a language built-in, there is an open question on how to make that work with custom allocators. Using capabilities would provide a compelling solution for that:</p> // Rust today </span>let</span> s = String::from("</span>Chashu</span>"); </span> </span>// With an `s` prefix, allocating on the global allocator </span>let</span> s = s"</span>Chashu</span>"; </span> </span>// With an `s` prefix, allocating on a local allocator </span>with alloc = Bump::new() { </span> </span>let</span> s = s"</span>Chashu</span>"; </span>} </span></code></pre> Conclusion</h2> In this post we took a look at general allocation strategies, how allocations can be nested today, which problems that approach has, and how we could simplify that via the novel contexts/capabilities language-feature. The design of contexts/capabilities is incomplete, so we intentionally didn't go too deep into that. Instead we only touched on the top-level semantics, showing specifically how it could be leveraged to overcome some of the problems we have with allocation today.</p> Fundamentally capabilities don't provide anything new</em>; they're just a fancy way of passing arguments to functions. But I also believe that is their strength. It is just passing arguments to functions</em>. But with significantly less code to write, no method duplication, the ability to play along with trait impls, and a lot more clarity of what your function needs to function</a>.</p> I don't just think that capabilities could simplify the problem space for allocators in Rust. I actually believe that it could be the canonical way in which we solve the current core</code>/alloc</code>/std</code> split in Rust. The core</code> library really is just std</code> minus everything which relies on ambient capabilities. If we could opt-out of relying on ambient capabilities in std</code>, the reason to have separate core</code>, alloc</code>, and std</code> libraries should go away.</p> The other thought I want to share is that I believe capabilities such as allocators are inherently tree-structured</em>. I've recently written about tree-structured concurrency</a> explaining how concurrency is best thought of as a tree. But from the operational semantics side of the language we now tree-borrows</a> as well. I feel like at a high-level thinking about program execution as a tree is accurate, and we should be thinking of (nesting) capabilities such as allocators as trees as well.</p> Totality Thu, 20 Jul 2023 00:00:00 +0000 Recently Eric</a> and I have been chatting with Daan Leijen</a>, of Koka</a> fame, discussing among other things the distribution of effects. Daan estimated that for a typical program written in Koka the distribution of effects would roughly be something like:</p> ~70% of a program could be "total" (as in: has no effects)</li> ~15% of a program would have to deal with exceptions or divergence (may error, may panic, or may loop indefinitely)</li> ~5% of a program ends up dealing directly with IO resources</li> </ul> That leaves another ~10% for wiggle room and other effects. And I think the broad implications of this are fascinating</em>. At least for Koka, according to Daan the vast majority of code could be written as entirely un-effectful. No panics, no infinite loops, no IO - just functions operating on input and producing output.</p> total functions</h2> Perhaps you may have heard people refer to Haskell as a "pure" language. Functions in Haskell may by default diverge (loop infinitely), and they may throw exceptions. Koka on the contrary is what you could call a "total language". By default functions are considered "total</em>" 1</a></sup>, and are only allowed to operate on their inputs, and produce outputs. In Koka if you want to write a "pure" function, you write a function which makes use of the exn</code> effect ("may throw exceptions"</em>) and div</code> effect ("may diverge"</em>):</p> ^{1</sup> I believe terminology here is plucked from math/category theory, which I know little about - so I hope I'm doing this description justice.</p> </div>}// Has no side-effects ("total") </span>fun </span>square_total</span>(</span>x</span>: </span>int</span>): </span>total int </span>{ </span> </span>x </span> </span>x </span>} </span> </span>// Doesn't terminate + throws exceptions ("pure") </span>// "pure" is an alias for the `<div,exn>` effects </span>fun </span>square_pure</span>(</span>x</span>: </span>int</span>): </span>pure int </span>{ </span> </span>throw</span>("</span>oops</span>"); </span> </span>x </span>* </span>square_pure</span>(</span>x</span>) </span>} </span></code></pre> In Rust however, we can't write total</em> functions. While we are able to write functions which meet all the conditions to be total</em> - we can't actually express in the type system that they're total. Meaning that by reading a function signature you have no idea whether a function will terminate, may panic, or perhaps open files - it doesn't say.</p> We get pretty close to the "pure" set of effects through the const fn</code> notation. It can't heap-allocate, it can't access IO, and can't access globals ("statics"). Right now it's also deterministic</em> - in Koka if code wants to be nondeterministic it needs to be marked as ndet</code></a>. But in order to declare "total" functions, we'd need to be able to strip the set effects on functions further in Rust beyond what const fn</code> does:</p> The ability to declare functions which will not panic</li> The ability to declare function which "will not diverge" (guarantee termination)</li> </ol> So let's take a look at what that means, and what it would take to implement in Rust.</p> Functions which cannot panic</h2> Rust distinguishes between "panic" for programmer-errors and "result" for runtime-errors. Both can be mapped into each other using a combination of:</p> </th> result</th> panic</th></tr></thead> Create</td> return Err(..)</code></td> panic_any</code></a> / unwrap</code></a></td></tr> Consume</td> match {}</code></td> catch_unwind</code></a></td></tr> </tbody></table> However, despite it being possible to convert between panics and errors - panics themselves do not exist anywhere in the Rust type system. You cannot construct a panic type and pass it around, you can only trigger a panic in-place. Similarly functions don't return a Panic<T></code> or impl Panic</code> when they can panic. Every function can panic, and the ability to panic lives outside of the type system. 2</a></sup></p> ^{2</sup> I don't think it's necessarily bad that panics live outside of the type system though. I'm definitely not convinced that we should bring it into the type system? Maybe it's closer to const</code> in the way that it just annotates the function, but it doesn't live in</em> the type system? idk.</p> </div>}"But Yosh?", I hear you asking, "Do people even want</em> to write functions which can't panic?" And well dear reader, people certainly want to be able to do that. There's a thread full of people here</a> discussing wanting the ability to opt-out of panics. For reasons such as better FFI (unwinding through FFI boundaries is generally unsound in Rust), support for embedded systems, and more. Formal verification tools such as Prusti</a> and Kani</a> advertise their ability to prove the absence of panics as their key features. And I know some folks working on Windows here at Microsoft who would love it if we could prove that code which should not panic in fact will never panic.</p> A basic form of "will not panic" notations can be implemented today via third party crates such as Dtolnay's no-panic</a>, and Japaric's panic-never</a>. But while it works okay, code like that is not composable and will eventually run into the effect sandwich problem</a>. Instead the better approach is probably to bring it into the type system, and reason about it as an effect.</p> While a "do not panic" notation is sufficient to prove that a function won't panic, inside the function body we're then obliged to actually write code which won't panic. And even with notations that's hard because of a number of reasons:</p> In Rust any operation may currently panic - opting into "will not panic" semantics would be a pretty big shift.</li> Proving panic-freedom in a way that is also ergonomic requires introducing verification tools into the language. Pattern types</a> backed by something like Flux</a> could be one approach. But this is an incredibly rich topic with lots of challenges - many of which we don't know if we can overcome.</li> </ul> In my post on pattern types</a> I go into this with some more detail on how we could deal with the first issue. Roughly said: if we were able to encode more details about the types in the types themselves, we could use that to prove to the compiler certain operations are fine:</p> // ❌ A basic doubling function in today's Rust </span>// combined with a "no_panic" notation </span>// </span>// Compiler error: this could panic because: u32::MAX + u32::MAX >= u32::MAX </span>#[</span>no_panic</span>] </span>fn </span>double</span>(</span>num</span>: </span>u32</span>) -> </span>u32 </span>{ </span> num + num </span>} </span> </span>// ✅ A basic doubling function using a hypothetical "pattern type" </span>// notation combined with a "no_panic" notation </span>const</span> N: </span>u32 </span>= </span>u32</span>::</span>MAX </span>/ </span>2</span>; </span>#[</span>no_panic</span>] </span>fn </span>double</span>(</span>num</span>: </span>u32</span>[0..N]) -> </span>u32</span>[</span>0</span>..</span>u32</span>::</span>MAX</span>] { </span> num + num </span>} </span></code></pre> functions which guarantee termination</h2> In Rust by default functions do not guarantee termination. And that's not a bad property: we want</em> to be able to write functions which don't guarantee termination, such as HTTP servers which keep listening indefinitely for requests, or other similar "main loops". Useful stuff.</p> But if every part of your program can loop indefinitely, it acts a little bit like if any part of your program has access to raw pointers or IO: it can be hard to track where exactly things are going wrong, because the potential site of it is undetermined.</p> Just like with the hypothetical "no_panic" notation, we could have some form of a "nobody_loops" 3</a></sup> notation. This could signal to the compiler that a function will guarantee termination, and that actions which don't terminate are disallowed. This one is undecidedly hard</del> impossible to prove for the general case, but if we wanted to do something practical we could probably create a new unsafe marker trait like say BoundedIterator</code> which would be automatically implemented for things like looping over ranges, looping over items in collections, etc.</p> ^3</sup>https://github.com/rust-lang/rust/pull/19964</a></p> </div> // ✅ This would compile fine since the loop in the body has an upper bound </span>#[</span>nobody_loops</span>] </span>fn </span>main</span>() { </span> </span>for</span> n in </span>0</span>..</span>100 </span>{ </span> println!("</span>{n}</span>"); </span> } </span>} </span></code></pre> Similar to Rust's borrow-checking rules, it's unlikely any implementation for "will terminate" will be enough to ever allow the full set of hypothetically compliant programs. That requires proving termination in the general case, which I believe may be impossible. Instead the focus should be on a practical</em> subset of programs which can provably terminate. This is another potential reason why generators as a first-class language feature would be useful. Because the compiler generates the iterator implementation, it would enable creating terminating iterators without manual unsafe trait impls:</p> // Made-up syntax. This would return: </span>// -> impl Iterator<Item = Cat> + BoundedIterator </span>#[</span>nobody_loops</span>] gen </span>fn </span>cats</span>() -> Cat { .. } </span></code></pre> Const Functions</h2> In Rust the const</code> effect isn't so much an effect onto itself, as it is the subtraction of effects from the "base" set of effects. As we mentioned: const functions must (currently) be deterministic, don't get access to IO, don't get access to statics, and can't allocate on the heap. And in return they gain the ability to be evaluated during compilation.</p> But that's if we view it from the perspective of Rust's base set of effects. If we look at const from a "total" perspective, then "const" represents roughly represents the "may panic" and "may diverge" effects. And while we don't have any annotations for basic functions in Rust, they effectively act as a superset of "const", and also gain access to globals, the host environment (io), etc.</p> So if we take a position that const functions somehow subtract effects from "base Rust", then it is indeed the case that "const" can probably not be considered an effect. But if we take a look at the effects from a perspective of totality, then "const" and "base" contexts simply represent different sets of effects:</p> total</strong>: no effects</li> const</strong>: panic + diverge</li> base</strong> 4</a></sup>: panic + diverge + allocate + host environment + globals</li> </ul> ^{4</sup> By "base Rust" I mean the capabilities a function has when you write fn foo</code> with no other annotations, compiled for a std-capable target. It doesn't have any specific effect annotations like const</code> or async</code>, hence the "base" qualifier.</p> </div> Reasoning about effects using a starting point of totality is also far simpler. It might not necessarily map to the surface-level syntax Rust provides, but it certainly makes it easier to reason about what happens when say, you have an unsafe const fn</code> compared to a async gen fn</code> or something. Both have added effects on top of "total", but while they slightly overlap they also differ somewhat. This makes it easier to reason about than when only talking about effect in terms of subsets or supersets, rooted in "base" Rust.</p>}Evolving Totality</h2> So far we've mainly focused at the "may panic" and "does not guarantee termination" effects. We've made the case that if only we had both those effects, we'd achieve totality. But that's not quite true. What "total" means for a language will differ depending on the language. In a way you think of "total" as the absence of all optional effects, but it includes all non-optional assumptions.</p> Over time we can discover we want more control over which assumptions are baked in and are optional. We've seen this with "const rust" already, which provides fewer effects than "base rust" does. And both "may panic" and "may diverge" are built-in assumptions which exist in Rust today, but could potentially be surfaced. But those aren't the only ones, there are potentially more such as:</p> ⚠️ Reader alert: I'm not proposing we add any of these effects. These are just intended as illustrative examples for a point I'm working up towards the end of this section. ⚠️</em></p> Access to host environment</strong>: every function in Rust assumes it has access to a host environment, and it can just call things like File::open</code> to access resources on the host. Embedded systems can't assume a host. The difference between the two is currently not encoded in the type system, and a notion of "total" may want to do away with the assumption a host is present.</li> Drop is not guaranteed to be called</strong>: today any function in Rust can freely take any type and pass it to mem::forget</code> or similar to leak them without calling their destructors. In a way you can think of every function having access to a leak effect</a>, and a notion of "total" may want to exclude that.</li> Type conversions can be reversed</strong>: say a function takes an impl Read</code> argument, using the Any</code> trait</a> it's possible to obtain the underlying type, as long as the type name is accessible. This means there is no guarantee that a function will only operate on the impl Read</code> interface, because it may in fact cast it back to the original type and operate on that instead. Being able to obtain the underlying type could be considered an effect which "total" may not want to have access to.</li> Type sizes can be accessed</strong>: every type has a size and a layout. Functions can learn information about a type by calling the size_of</a> function on a type. A notation of totality could restrict obtaining any size or layout information about a type.</li> Memory locations can be directly accessed</strong>: we assume types are backed by memory addresses in the address space. This means we can always find the memory address of a type. But what if we were to run, say, Rust on the JVM. The JVM doesn't expose an address space; it exposes objects and references. Which means that "type locations can be accessed" could be considered an effect, which could be absent from totality by default.</li> </ul> And there are probably lots more potential "effects" which fall outside of the notion of "totality" presented in this post so far. That's because in a way writing effects brings the assumptions we make about functions directly into the type system. And because our programs run on real physical hardware with memory models and operating systems, it's possible to map just about any detail of the underlying hardware into an effect. Even to the point of silliness, such as an effect which enables you to allocate stack space.</p> Choosing which effects to add means choosing which assumptions to surface. That is a real art, and requires thinking through the purpose and use cases of the effects. Ideally effects could even come in themes: for example "type conversions can be reversed" and "type sizes can be accessed" share a theme of "type privacy" - maybe they could be folded into each other. We've done something similar with "const" too, where we've combined "panic" and "diverge" into a single effect. What I'm trying to say is: what we mean by "total" depends on our current idea of what the language is and should do. And it's even something which can change over time as our model and needs of the language evolve.</p> Conclusion</h2> Total functions go a step beyond "pure" functions because they also disallow code from panicking and require functions provably terminate. Effect-wise it represents the least a function could possibly do, so when talking about effects we can think of every other effect as "adding" effects on top of the "total" effect. This will also be helpful in case we ever manage to constify</em> the majority of the stdlib, and want to switch the default set of implied effects across an edition bound. Reasoning about effects rooted in totality provides us with a model to reason about what would change.</p> As we're closing out here, something I did want to briefly touch on Rice's theorem</a>. It states that: "All non-trivial semantic properties of programs are undecidable".</em> In effect what we're doing with effects is attempting to go against that. The unsafe</code> keyword in Rust exists because the compiler can't prove memory safety for all</em> functions, just most of them but we need a way out for when we can't. Similarly we won't be able to prove panic-freedom for all functions, or prove all functions can terminate. But we can for a lot of functions; and statically proving interesting properties of our programs is kind of what type systems are about.</p> I find it fascinating that ~70% of code in a typical Koka program can be marked as total. For Rust there is a thriving research movement to build verifiers for the language, and one of the properties they often start by proving is panic-freedom for functions. I can't help but wonder how we could not just expose these kinds of guarantees through extensions to Rust, but actually by</em> Rust itself. And I think in order to do that, we may want to start with by adding ways to reason about semantic properties such as "may panic" or "may diverge" as effects native to Rust.</p> Thanks to Eric Holk</a> and Daan Leijen</a> for reviewing this post prior to publishing, and chatting about effects over the past few months.</em></p> bridging fuzzing and property testing Mon, 10 Jul 2023 00:00:00 +0000 It's been over three years since Fitzgen published: "Announcing Better Support for Fuzzing with Structured Inputs in Rust"</em></a>, and a little over two years since arbitrary</a> 1.0 was released. A few years agoI wrote a property-testing crate based on arbitrary</code>, but never ended up writing about it. So I wanted to take a moment to change that. We'll start by taking a look at automated testing, then cover the different approaches to it, explain how to work with structured inputs, and finally introduce heckcheck</a> - a small property-testing library.</p> Update 2023-07-18:</strong> I recently learned about a different crate using a similar approach to mine: proptest-arbitrary-interop</a>. The proptest crate is more mature than my heckcheck</code> project, so if you're considering adopting the techniques described in this post - please check out the proptest interop crate too.</p> fuzzing vs property testing</h2> Both fuzzing</a> and property testing</a> are ways of automatically testing code. Where unit tests typically test some expected set of behavior, automated test have the ability to be more rigorous and exhaustive - making them far more likely to weed out bugs - especially those you didn't even conceive of. Fuzzing and property testing have a lot in common with each other. Where they tend to differ is how tests are driven.</p> With fuzzing you typically use some external agent to test your program. Fuzzers usually have the ability to instrument the code under test, and make use of tools such as sanitizers</a> to check whether invariants are violated. Fuzzers will also keep track of which input leads to which branches being hit in code, tailor their input to cover as many branches as possible. This process can take time, and is why it often pays off to run fuzzers for extended periods of time or even continuously</a>. In Rust support for fuzzing is provided via the cargo-fuzz</code></a> extension.</p> Property testing typically works the other way around: property testing is typically done by including a library in your test code which allows you to drive the tests yourself. Rather than looking for crashes or sanitizer failures the emphasis is more on checking your implementation using a strategy - more on those in a later section. While fuzzing tends to be more thorough, property testing tends to be faster to execute. Depending on what you're testing it's not uncommon to integrate property tests in your CI test suite - and run them every time you make a change. If you're writing unsafe Rust you can even combine property tests with miri</a> to validate with more confidence whether the runtime behavior of your program matches Rust's operational semantics.</p> The final piece of both property testing and fuzzing is that they perform something called test case shrinking</em> when they're done. When they encounter a failing test, what they'll do is try and simplify the input to the bare minimum to isolate the issue as much as possible. Once simplified it tends to be much easier to understand, and in turn can become easier to convert into unit tests</a>.</p> There's a lot more to say about automated testing - including how to use it to mess with the ordering of concurrency primitives</a>, or to intentionally mess with the network</a>. But roughly speaking that should all just be a matter of wiring up drivers up to the same source of entropy coming from the automated testing drivers.</p> Structured Inputs</h2> In Fitzgen's post</a> he shows how it's possible to combine fuzing with structured inputs using arbitrary</code> crate. The way a fuzzer typically works is that it generates random data which is passed to a program over some channel. Sometimes this data is generated from some corpus of data to try and weave in phrases we know the program might be looking for - such as HTTP/1.1</code> or GET</code> for an HTTP parser. But with arbitrary</code> this randomness can be more clearly channeled: it can take the arbitrary stream of data, and use it to create structured types.</p> Part of the reason why Fitzgen did this work was to fuzz WASM programs. In "Writing a Test Case Generator for a Programming Language"</em></a> he describes how using cargo fuzz</code> and arbitrary</code> he's able to generate endless amounts of new, valid WASM programs to pass to be used to test all sorts of WASM-related tools</a>. This allows far more code to be exercised than solely using a corpus-based approach ever could.</p> To show an example of what it looks like to use cargo-fuzz, we can re-use the RGB parser/encoder example from Fitzgen's blog - but with the actual methods implemented. We start by defining a struct which takes red, green, and blue values:</p> use </span>arbitrary::Arbitrary; </span> </span>/// A color value encoded as Red-Green-Blue </span>#[</span>derive</span>(Arbitrary, Debug, PartialEq, Eq)] </span>pub struct </span>Rgb { </span> </span>r</span>: </span>u8</span>, </span> </span>g</span>: </span>u8</span>, </span> </span>b</span>: </span>u8</span>, </span>} </span> </span>impl </span>Rgb { </span> </span>/// Convert from RGB to Hexadecimal. </span> </span>pub fn </span>to_hex</span>(&</span>self</span>) -> String { </span> format!("</span>#</span>{:02X}{:02X}{:02X}</span>", </span>self</span>.r, </span>self</span>.g, </span>self</span>.b) </span> } </span> </span> </span>/// Convert from Hexadecimal to RGB. </span> </span>pub fn </span>from_hex</span>(</span>s</span>: String) -> </span>Self </span>{ </span> </span>let</span> s = s.</span>strip_prefix</span>('</span>#</span>').</span>unwrap</span>(); </span> Rgb { </span> r: </span>u8</span>::from_str_radix(&s[</span>0</span>..</span>2</span>], </span>16</span>).</span>unwrap</span>(), </span> g: </span>u8</span>::from_str_radix(&s[</span>2</span>..</span>4</span>], </span>16</span>).</span>unwrap</span>(), </span> b: </span>u8</span>::from_str_radix(&s[</span>4</span>..</span>6</span>], </span>16</span>).</span>unwrap</span>(), </span> } </span> } </span>} </span></code></pre> We can expose this to cargo-fuzz</code> to perform a roundtrip-test by creating a new file containing the following:</p> // The fuzz target takes a well-formed `Rgb` as input! </span>libfuzzer_sys::fuzz_target!(\|</span>rgb</span>: Rgb\| { </span> </span>let</span> hsl = rgb.</span>to_hsl</span>(); </span> </span>let</span> rgb2 = hsl.</span>to_rgb</span>(); </span> </span> </span>// RGB -> HSL -> RGB is the identity function. This </span> </span>// property should hold true for all RGB colors! </span> assert_eq!(rgb, rgb2); </span>}); </span></code></pre> We can then execute it using cargo fuzz</code> by running:</p> $</span> cargo fuzz run my_test </span></code></pre> Heckcheck: Property Testing using Arbitrary</h2> As we've covered the arbitrary</code> crate works great when fuzzing. And because fuzzing and property testing are very similar, you'd assume that it should be possible to use arbitrary</code> for property-based tests too right? I couldn't find a crate which did this, so I ended up writing one! Enter heckcheck</a>, a really small property-testing library. If we take our earlier RGB example and adapt it to use heckcheck, it will look something like this:</p> /// Validate values can be converted from RGB to Hex and back. </span>#[</span>test</span>] </span>fn </span>rgb_roundtrip</span>() { </span> heckcheck::check(\|</span>rgb</span>: Rgb\| { </span> </span>let</span> hex = rgb.</span>to_hex</span>(); </span> </span>let</span> res = Rgb::from_hex(hex); </span> assert_eq!(rgb, res); </span> Ok(()) </span> }); </span>} </span></code></pre> And that's all we had to do. To test the code we just run cargo test</code> as usual:</p> $</span> cargo test </span></code></pre> The same code we would've written to interact with a fuzzer can now be used for property testing too. As I've said earlier: both fuzzing and property testing have more in common than not. The difference here compared to using a fuzzer is that this can be run as part of CI without a problem, executes really quickly, and is entirely portable. Fuzzers need to be installed on the system first, typically execute more slowly, and some platforms such as Windows have limited support for fuzzers.</p> I wrote heckcheck</code> mostly to see whether it would be possible to write - and the implementation is maybe ~400 lines or so in total. I bet it could be improved, since in particular our shrinking code is less than ideal, while good approaches are known</a>.</p> Compared to other popular property testing libraries heckcheck</code> is more similar to quickcheck</code></a> than it is to proptest</code></a>. It has a simple interface, can generate complex values quickly, and uses external shrinking strategies. The proptest docs</a> contain a comparison between proptest and quickcheck, and that same comparison could probably apply between proptest and heckcheck. The only place where the comparison may differ is that the arbitrary</code> crate has support for customizing generated values. Say we wanted to restrict the values generated in our RGB implementation to only valid RGB values, we could change the implementation to the following:</p> use </span>arbitrary::{Arbitrary, Unstructured}; </span> </span>/// A color value encoded as Red-Green-Blue </span>#[</span>derive</span>(Arbitrary, Debug, PartialEq, Eq)] </span>pub struct </span>Rgb { </span> #[</span>arbitrary</span>(with = color_range)] </span> </span>r</span>: </span>u8</span>, </span> #[</span>arbitrary</span>(with = color_range)] </span> </span>g</span>: </span>u8</span>, </span> #[</span>arbitrary</span>(with = color_range)] </span> </span>b</span>: </span>u8</span>, </span>} </span> </span>/// Generate a valid RGB color value </span>fn </span>color_range</span>(</span>u</span>: &</span>mut</span> Unstructured) -> arbitrary::Result<</span>u8</span>> { </span> u.</span>int_in_range</span>(</span>0</span>..=</span>255</span>) </span>} </span></code></pre> If I could change one thing about the arbitrary</code> crate it's that I'd like to see it support pattern-type notation inside the attributes. Patterns allow primitive values to be restricted to a subset of their possible inputs, which should be a fairly common use. With that we could imagine the RGB type could instead be written like this:</p> //! ⚠️ This is purely speculative and not currently supported ⚠️ </span> </span>use </span>arbitrary::{Arbitrary, Unstructured}; </span> </span>/// A color value encoded as Red-Green-Blue </span>#[</span>derive</span>(Arbitrary, Debug, PartialEq, Eq)] </span>pub struct </span>Rgb { </span> #[</span>arbitrary</span>(pattern = 0..=255)] </span> </span>r</span>: </span>u8</span>, </span> #[</span>arbitrary</span>(pattern = 0..=255)] </span> </span>g</span>: </span>u8</span>, </span> #[</span>arbitrary</span>(pattern = 0..=255)] </span> </span>b</span>: </span>u8</span>, </span>} </span></code></pre> Automated testing strategies</h2> Now that we've covered what the different ways of automated testing are, it's worth briefly touching on some common testing strategies. Because once you've understood that automated testing might be good, there often is a bit of a gap to actually implement it in your own code. And that's what testing strategies are for. These are some common testing strategies:</p> roundtrip testing</strong>: generate a message, pass it to the encoder, then pass the encoder's output to the decoder. The decoder's output should be the same as the original message.</li> differential testing</strong>: test the program against a "known good" implementation of a similar program (also known as an "oracle"). This is useful if, for example, you're re-implementing an existing protocol or algorithm.</li> invariant testing</strong>: test that a certain property always</em> holds. This can be a universal property like: "my program didn't crash". But invariants can be specific too. Rain</a> recently gave a great example that if you're for example testing a sorting function, you can check its correctness by making iterating over each number and checking it's bigger than the last number.</li> </ul> We've already covered roundtrip testing earlier in this post, and I believe invariant testing should be fairly straight forward. So as we're closing out here I just briefly wanted to show an example of how to perform differential testing using an approach Tyler Neely</a> introduced me to a few years ago.</p> The way this works is that you use your source of entropy to generate a sequence of operations which you then apply to both your oracle and your program under test. Should your test trigger a failure, the shrinker will kick in and try and remove operations trying to uncover the minimal sequence of operations needed to trigger the bug. Say we in the process of building the smallvec</a> crate; the implementation differs from the stdlib, but the observable behavior should be the same. Which means we can use the stdlib's Vec</code> as our oracle for the test, which could look something like this:</p> /// A single operation we can apply </span>#[</span>derive</span>(Arbitrary)] </span>enum </span>Operation { </span> Insert(</span>usize</span>, </span>usize</span>), </span> Fetch(</span>usize</span>), </span>} </span> </span>#[</span>test</span>] </span>fn </span>differential_smallvec_test</span>() { </span> heckcheck::check(\|</span>operations</span>: Vec<Operation>\| { </span> </span>// Setup both our subject and the oracle </span> </span>let mut</span> subject = smallvec::SmallVec::new(); </span> </span>let mut</span> oracle = vec![]; </span> </span> </span>// Apply the same operations in-order to both the subject and the oracle </span> </span>// comparing outputs whenever we get any. </span> </span>for</span> operation in operations { </span> </span>match</span> operation { </span> Insert(index, value) => { </span> oracle.</span>insert</span>(index, value); </span> subject.</span>insert</span>(index, value); </span> } </span> Fetch(index) => { </span> assert_eq!(oracle.</span>get</span>(index), subject.</span>get</span>(index)); </span> } </span> } </span> } </span> }); </span>} </span></code></pre> Conclusion</h2> In this post we've covered what fuzzing and property testing are, how they're similar, and how they differ. We've introduced heckcheck</a>, a small property-testing library and shown how you can use it to implement various testing strategies with.</p> I believe one of the greatest strengths Rust has is its ability to provide uniform experiences. And I believe that automated testing is one of the best tools we have to ensure the correctness of the implementations of our Rust programs. I'd love to one day see Rust provide facilities out of the box for automated testing - including a standard Arbitrary</code> interface.</p> Tree-Structured Concurrency Sat, 01 Jul 2023 00:00:00 +0000 For a while now I've been trying to find a good way to explain what structured concurrency is, and how it applies to Rust. I've come up with zingers such as: "Structured concurrency is structured programming as applied to concurrent control-flow primitives"</em>. But that requires me to start explaining what structured programming is, and suddenly I find myself 2000 words deep into a concept which seems natural to most people writing programs today 1</a></sup>.</p> Instead I want to try something different. In this post I want to provide you with a practical introduction to structured concurrency. I will do my best to explain what it is, why it's relevant, and how you can start applying it to your rust projects today. Structured concurrency is a lens I use in almost all of my reasoning about async Rust, and I think it might help others too. So let's dive in.</p> This post assumes some familiarity with async Rust</a> and async cancellation</a>. If you aren't already, it might be helpful to skim through the earlier posts on the topic.</em></p> ^1</sup>If you're interested in structured programming though, I can recommend Dijkstra's writing</a> on the subject. Most programming languages we use today are structured, so reading about a time when when that wasn't the case is really interesting.</p> </div> What is structured concurrency?</h2> Structured concurrency is a property of your program. It's not just any structure, the structure of the program is guaranteed to be a tree</a> regardless of how much concurrency is going on internally 2</a></sup>. A good way to think about it is that if you could plot the live call-graph of your program as a series of relationships it would neatly form a tree. No cycles 3</a></sup>. No dangling nodes. Just a single tree.</p> ^{2</sup> When I was researching this topic last year I found a paper from I believe the 80's which used the phrase "tree-structured concurrency". I couldn't find it again in time for this post, but I remember tweeting about it because I hadn't seen the term before and I thought it was really helpful!</p> </div>}^{3</sup> Yes yes, recursion can make functions call themselves - or even call themselves through a proxy. By "live call-graph" I mean it like how flame charts visualize function calls. Recursion is visualized by stacking function calls on top of each other. The same idea would apply here by adding new async nodes as children of existing nodes. The emphasis here is very much on live</em> call-graph, not logical</em> call-graph.</p> </div>} Fig 1. The arrows point from parent nodes to child nodes. It has no cycles. A parent can have multiple children. But a child always has a single parent - except for the root node. </figcaption> </figure> And this structure, at least in async Rust, provides three key properties:</p> Cancellation propagation:</strong> When you drop a future to cancel it, it's guaranteed that all futures underneath it are also cancelled.</li> Error propagation:</strong> When an error is created somewhere down in the call-graph, it can always be propagated up to the callers until there is a caller who is ready to handle it.</li> Ordering of operations:</strong> When a function returns, you know it is done doing work. No surprises that things are still happening long after you thought the function had completed.</li> </ul> These properties put together lead to something called a "black box model of execution"</strong>: under a structured model of computing you don't need to know anything about the inner workings of the functions you're calling, because their behavior is guaranteed. A function will return when it's done, will cancel all work when you ask it to, and you'll always receive an error if there is something which needs handling. And as a result code under this model is composable</strong>.</p> Fig 2. Under structured concurrency every future has a parent, cancellation flows downward, and errors flow upward. When a future returns, you can be sure it's done doing work. </figcaption> </figure> If your model of concurrency is unstructured</em>, then you don't have these guarantees. So in order to guarantee that say, cancellation is correctly propagated, you'll need to inspect the inner workings of every function you're calling. Code under this model is not composable, and requires manual checks and bespoke solutions. This is both labor-intensive and prone to errors.</p> Unstructured concurrency: an example</h2> Let's start by implementing a classic concurrency pattern: "race". But rather than using structured</em> primitives, we can use the staples of unstructured programming: the venerable task::spawn</code> and channel</code>. The way "race" works is it takes two futures, and we try and get the output of whichever one completes first is whose message we read. We could write it something like this:</p> use </span>async_std::{channel, task}; </span> </span>let </span>(sender0, receiver) = channel::bounded(</span>1</span>); </span> </span>let</span> sender1 = sender0.</span>clone</span>(); </span>task::spawn(async </span>move </span>{ </span>// 👈 Task "C" </span> task::sleep(Duration::from_millis(</span>100</span>)); </span> sender1.</span>send</span>("</span>first</span>").await; </span>}); </span> </span>task::spawn(async </span>move </span>{ </span>// 👈 Task "B" </span> task::sleep(Duration::from_millis(</span>100</span>)); </span> sender0.</span>send</span>("</span>second</span>").await; </span>}); </span> </span>let</span> msg = receiver.</span>recv</span>().await; </span>// 👈 Future "A" </span>println!("</span>{msg}</span>"); </span></code></pre> While this implements "race" semantics correctly, it doesn't handle cancellation. If one of the branches completes, we'd ideally like to cancel the other. And if the containing function is cancelled, both computations should be cancelled. Because of how we've structured the program neither task is anchored to a parent future, and so we can't cancel either computation directly. Instead the solution would be to come up with some design using more channels, anchor the handles - or we could instead rewrite this using structured primitives.</p> Fig 3. You can create a "race" operation by combining tasks and channels. Data can flow out of the tasks to the caller. But because the tasks aren't rooted in a parent task, cancellation doesn't propagate. </figcaption> </figure> Structured concurrency: an example</h2> We can rewrite the example above using structured primitives instead. Rather than DIY-ing our own "race" implementation using tasks and channels, we should instead be using a "race" primitive which implements those semantics for us - and correctly handles cancellation. Using the futures-concurrency</a> library we could do that as follows:</p> use </span>futures_concurrency::prelude::; </span>use </span>async_std::task; </span> </span>let</span> c = async { </span>// 👈 Future "C" </span> task::sleep(Duration::from_millis(</span>100</span>)); </span> "</span>first</span>" </span>}; </span> </span>let</span> b = async { </span>// 👈 Future "B" </span> task::sleep(Duration::from_millis(</span>100</span>)); </span> "</span>second</span>" </span>}; </span> </span>let</span> msg = (c, b).</span>race</span>().await; </span>// 👈 Future "A" </span>println!("</span>{msg}</span>"); </span></code></pre> When one future completes here, the other future is cancelled. And should the Race</code> future be dropped, then both futures are cancelled. Both futures have a parent future when executing. Cancellation propagates downwards. And while there are no errors in this example, if we were working with fallible operations then early returns would cause the future to complete early - and errors would be handled as expected.</p> Fig 4. By using a structured "race" primitive all child futures are rooted in a parent future. Which allows both cancellation and errors to propagate. And the operation won't return until all child futures have dropped. </figcaption> </figure> So far we've looked at just the "race" operation, which encodes: "Wait for the first future to complete, then cancel the other"</em>. But other async concurrency operations exist as well, such as:</p> join</strong>: wait for all futures to complete.</li> race_ok</strong>: wait for the first future to complete which returns Ok</code>.</li> try_join</strong>: wait for all futures to complete, or return early if there is an error.</li> merge</strong>: wait for all futures to complete, and yield items from a stream as soon as they're ready.</li> </ul> There are a few more such as "zip", "unzip", and "chain" - as well as dynamic concurrency primitives such as "task group", "fallible task group", and more. The point is that the set of concurrency primitives</em> is bounded. But they can be recombined in ways that makes it possible express any form of concurrency you want. Not unlike how if a programming language supports branching, loops, and function calls you can encode just about any control-flow logic you want,- without ever needing to use "goto".</p> What's the worst that can happen?</h2> People sometimes ask: What's the worst that can happen when you don't have structured concurrency? There are a number of bad outcomes possible, including but not limited to: data loss, data corruption, and memory leaks.</p> While Rust guards against data races</em> which fall under the category of "memory safety", Rust can't protect you from logic bugs. For example: if you execute a write</code> operation inside of a task whose handle isn't joined, then you'll need to find some alternate mechanism to guarantee the ordering of that operation in relation to the rest of the program. If you get that wrong you might accidentally write to a closed resource and lose data. Or perform an out-of-order write, and accidentally corrupt a resource 4</a></sup>. These kinds of bugs are not in the same class as memory safety bugs. But they are nonetheless serious, and they can be mitigated through principled API design.</p> ^{4</sup> At a previous job we experienced exactly this in a database client: we were having issues propagating cancellation correctly, which meant that the connection protocol could be corrupted because we didn't flush messages when we should have.</p> </div>}Applying structured concurrency to your programs</h2> task::spawn</strong></p> When using or authoring async APIs in Rust, you should ask yourself the following questions to ensure structured concurrency:</p> Cancellation propagation</strong>: If this future or function is dropped, will cancellation propagate to all child futures?</li> Error propagation</strong>: If an error happens anywhere in this future, can we either handle it directly or surface it to the caller?</li> Ordering of operations</strong>: When this function returns, will no more work continue to happen in the background?</li> </ol> If all of these properties are true, then once the function exits it's done executing and you're good. This however leads us to a major issue in today's async ecosystem: neither async-std nor tokio provide a spawn</code> function which is structured. If you drop a task handle the task isn't cancelled, but instead it's detached and will continue to run in the background. This means that cancellation doesn't automatically propagate across task boundaries, causing it to be unstructured.</p> The smol</a> library gets closer though. It has a task implementation which gets us closer to "cancel on drop"-semantics out of the box. Though it doesn't get us all the way yet because it doesn't guarantee an ordering of operations. When a smol Task</code> is dropped the task isn't guaranteed to have been cancelled, all it guarantees is that the task will be cancelled at some point in the future.</p> async drop</strong></p> Which brings us to the biggest piece missing from async Rust's structured concurrency story: the lack of async Drop in the language. Smol's tasks have an async cancel</a> method which only resolves once the task has successfully been cancelled. Ideally we could call this method in the destructor and wait for it. But in order to do that today we'd need to block the thread, and that can lead to throughput issues. No, in practice what we really need for this to work well is async destructors 5</a></sup>.</p> ^5</sup>Async cancellation is hardly the only motivation for async Drop. It also prevents us from encoding basic things like: "flush this operation on drop"</a> - which is something we can</em> encode in non-async Rust today.</p> </div> what can you do today?</strong></p> But while we can't yet trivially fulfill all requirements for async structured concurrency for async tasks, not all hope is lost. Without async Drop we can already achieve 2/3 of the requirements for task spawning today. And if you're using a runtime other than smol, adapting the spawn method</a> to work like smol's does is not too much work. But most concurrency doesn't need tasks because it isn't dynamic. For that you can take a look at the futures-concurrency</a> library which implements composable primitives for structured concurrency.</p> If you want to adopt structured concurrency in your codebase today, you can start by adopting it for non-task-based concurrency. And for task-based concurrency you can adopt the smol model of task spawning to benefit from most of the benefits of structured concurrency today. And eventually the hope is we can add some form of async Drop to the language to close out the remaining holes.</p> Pattern: managed background tasks</h2> People frequently ask how they can implement "background tasks" under structured concurrency. This is used in scenarios such as an HTTP request handler which also wants to submit a piece of telemetry. Rather than blocking sending the response on the telemetry, it spawns a "background task" to submit the telemetry in the background, and immediately returns from the request. This can look something like this:</p> let mut</span> app = tide::new(); </span>app.</span>at</span>("</span>/</span>").</span>post</span>(\|_\| async </span>move </span>{ </span> task::spawn(async { </span>// 👈 Spawns a background task… </span> </span>let</span> _res = </span>send_telemetry</span>(data, more_data).await; </span> </span>// … what if `res` is an `Err`? How should we handle errors here? </span> }); </span> Ok("</span>hello world</span>") </span>// 👈 …and returns immediately after. </span>}); </span>app.</span>listen</span>("</span>127.0.0.1:8080</span>").await?; </span></code></pre> The phrase "background task" seems polite and unobtrusive. But from a structured perspective it represents a computation without a parent - it is a dangling task</em>. The core of the pattern we're dealing with is that we want to create a computation which outlives the lifetime of the request handler. We can resolve this by rather than creating a dangling task to submit it to a task queue or task group which outlives the request handler. Unlike a dangling task, a task queue</em> or task group</em> preserves structured concurrency. Where a dangling task doesn't have a parent future and becomes unreachable, using a task queue we transmit the ownership of a future to a different object which outlives the current more ephemeral scope.</p> I've heard people make the argument before that task::spawn</code> is perfectly structured, as long as you think of it as spawning on some sort of unreachable, global task pool. But the question shouldn't be whether tasks are spawned on a task pool, but what the relationship is of those tasks to the rest of the program. Because we cannot cancel and recreate an unreachable task pool. Nor can we receive errors from this pool, or wait for all tasks in it to complete. It doesn't provide the properties we want from structured concurrency - so we shouldn't consider it structured.</p> I don't feel like the ecosystem has any great solutions to this yet - in part limited because we want "scoped tasks"</a> which basically require linear destructors</a> to function. But other experiments exist</a> so we can use that plus channels to put something together which gives us what we want:</p> ⚠️ Note: This code is not considered "good" by the author, and is merely used as an example to show that this is possible to write today. More design work is necessary to make this ergonomic ⚠️</em></p> // Create a channel to send and receive futures over. </span>let </span>(sender, receiver) = async_channel::unbounded(); </span> </span>// Create a structured task group at the top-level, next to the HTTP server </span>// </span>// If any errors are returned by the spawned tasks, all active tasks are cancelled </span>// and the error is returned by the handle. </span>let</span> telemetry_handle = async_task_group::group(\|</span>group</span>\| async </span>move </span>{ </span> </span>while let </span>Some(telemetry_future) = receiver.</span>next</span>().await { </span> group.</span>spawn</span>(async </span>move </span>{ </span> telemetry_future.await?; </span>// 👈 Propagate errors upwards </span> Ok(()) </span> }); </span> } </span> Ok(group) </span>}); </span> </span>// Create an application state for our HTTP server </span>#[</span>derive</span>(Clone)] </span>struct </span>State { </span> </span>sender</span>: async_channel::Sender<impl Future<Result<_>>>, </span>} </span> </span>// Create the HTTP server </span>let mut</span> app = tide::new(); </span>app.</span>at</span>("</span>/</span>").</span>post</span>(\|</span>req</span>: Request<State>\| async </span>move </span>{ </span> state.sender.</span>send</span>(async { </span>// 👈 Sends a future to the handler loop… </span> </span>send_telemetry</span>(data, more_data).await?; </span> Ok(()) </span> }).await; </span> Ok("</span>hello world</span>") </span>// 👈 …and returns immediately after. </span>}); </span> </span>// Concurrently execute both the HTTP server and the telemetry handler, </span>// and if either one stops working the other stops too. </span>(app.</span>listen</span>("</span>127.0.0.1:8080</span>"), telemetry_handle).</span>race</span>().await?; </span></code></pre> Like I said: we need to do a lot more API work to be able to rival the convenience of just firing off a dangling task. But what we lack for in API convenience, we make up for in semantics. Unlike our earlier example this will correctly propagates cancellation and errors, and every executing future is owned by a parent future. We could even take things a step further and implement things like retry-handlers with error quotas on top of this to create a more resilient system. But hopefully this is enough already to get the idea across of what we could be doing with this.</p> Guaranteeing Structure</h2> I've been asking myself for a while now: "Would it be possible for Rust to enforce structured concurrency in the language and libraries?"</em> I don't believe this is something we guarantee from the language. But it is something can</em> guarantee for Rust's library code, and make it so most async code is structured by default.</p> The reason why I don't believe it's fundamentally possible to guarantee structure at the language level is because it's possible to express any kind of program in Rust, which includes unstructured programs. Futures, channels, and tasks as they exist today are all just regular library types. If we wanted to enforce structure from the language, we would need to find a way to disallow the creation of these libraries - and that seems impossible for a general-purpose language 6</a></sup>.</p> ^{6</sup> An example of this from structured programming: Rust is a structured language. Assembly is not a structured language. You can implement an assembly interpreter entirely in safe Rust - meaning you can express unstructured code in a structured language. I could show examples of this, but eh I hope the general line of reasoning makes sense here.</p> </div> Instead it seems more practical to me to adopt tree-structured concurrency as the model we follow for async Rust. Not as a memory-safety guarantee, but as a design discipline we apply across all of async Rust. APIs which are unstructured should not be added to the stdlib. And our tooling should be aware that unstructured code may exist, so it can flag it when it encounters it.</p>}Conclusion</h2> In this post I've shown what (tree-)structured concurrency is, why it's important for correctness, and how you can apply it in your programs. I hope that by defining structured concurrency in terms of guarantees about propagation of errors and cancellation, we can create a practical model for people to reason about async Rust with.</p> As recently reported by Google</a>, async Rust is one of the most difficult aspects of Rust to learn. It seems likely that the lack of structure in async Rust code today did not help. In async code today neither cancellation nor errors are guaranteed to propagate. This means that if you want to reliably compose code, you need to have knowledge of the inner workings of the code you're using. By adopting a (tree-)structured model of concurrency these properties can instead be guaranteed from the outset, which in turn would make Async Rust easier to reason about and teach. Because "If it compiles it works"</em> should apply to async Rust too.</p> Thanks to Iryna Shestak for illustrating and proof-reading this post.</em></p> What is WASI? Fri, 02 Jun 2023 00:00:00 +0000 Introduction</h2> Now that the final touches are being put on the second version of WASI</a> (WASI Preview 2), I figured now might be a good time to write a few words on WASI, WebAssembly, and how to think about them. In recent months I've been driving the WASI Preview 2 work in the Rust compiler. So in order to do that I've had to familiarize where things are currently, where things are headed, and how build a mental model for myself. Because while Wasm and WASI have been around for a while, WASI especially has undergone a lot of changes. Enough so that I figured it might be helpful to write a little about.</p> What is WebAssembly?</h2> WebAssembly</a> is a bytecode</em> format which compilers can compile programs into. You can think of "bytecode" as an intermediate format which can be converted to native</em> formats such as x86 assembly</a>, which is what most desktop CPUs run, or ARM v8 assembly</a> which is what most mobile phones run. This is useful because it means you can "compile once" to WebAssembly Bytecode, and then run that code on any number of targets - provided they have a runtime which can interpret WebAssembly and convert it to native code.</p> WebAssembly was initially designed for web browsers, to provide a way to execute untrusted native code in a sandboxed environment. But this use case has since expanded, because it turns out that a high-performance trusted sandbox is useful abstraction for a lot of things - including creating sandboxed environments for networked applications.</p> The best way to think about the space WebAssembly runtimes occupy is roughly at the same level as "docker containers" or "VMs". But rather than virtualizing an entire operating system and managing that, WebAssembly runtimes use bytecode to virtualize single applications. Where a single computer may concurrently execute tens of VMs or hundreds of containers, it should be possible to run thousands of WebAssembly programs 1</a></sup>.</p> ^{1</sup> These are obviously not exact numbers, but more of a broad illustration of scale. I know that for example FireCracker VMs are more performant than regular VMs, and that people have successfully launched a million plus docker containers in contests. But those are not typical cases, and I believe in the typical</em> cases what I'm saying should be broadly accurate.</p> </div>}bytecode, WebAssembly, and the JVM</h2> WebAssembly is sometimes compared to Java and the JVM</a>. This makes sense, because the JVM is a popular platform which also interprets bytecode. The JVM not only supports Java but any language which compiles to JVM Bytecode, including Kotlin and Scala. However despite WebAssembly and the JVM both executing bytecode, there are some key differences between the two:</p> WebAssembly was designed to be targeted by native languages as C/C++/Rust in mind, while the JVM is more oriented towards 2</a></sup> garbage-collected languages.</li> WebAssembly is a royalty-free W3C standard</a>, while the JVM somewhat famously</a> isn't. This means implementing WebAssembly support in language toolchains and runtimes is not only possible, it's the exact goal of the project.</li> WebAssembly was designed from the ground up with with strict sandboxing as a core priority while the JVM wasn't. In order to enable secure multi-tenancy with the JVM it can be advisable to wrap it in another isolation layer - such as a VM.</li> </ol> ^2</sup>This changes a bit with the introduction of GraalVM</a>, but I think it still holds largely true.</p> </div> Note though that I don't mean to harsh the JVM at all with this. WebAssembly and the JVM were target very different use cases, have different priorities in their design, and in turn excel at very different things.</p> What are WebAssembly Components?</h2> WebAssembly as bytecode format is also often referred to as "Core WebAssembly". When you compile a "Core WebAssembly" program it is converted into something called a "WebAssembly Module". You can roughly of think of this as an object file in traditional compilation models. But unlike classic objects, Wasm modules can't describe system calls or reference any external symbols - they can only take numbers in and put numbers out, and that's about it. If you're trying to make Wasm programs do anything other than sum up numbers, you need more than just Core WebAssembly.</p> "WebAssembly Components" are a typed wrapper around Wasm Modules. Rather than reasoning about numbers in/numbers out, they provide a way to talk about types, functions, methods, and namespaces. The way this is done is via an IDL format</a> called WIT (Wasm Interface Types) </a>. Here's an example of a "monotonic clock" interface taken from the preview2-prototyping repo</a>:</p> default interface </span>monotonic-clock { </span> </span>use poll</span>.</span>poll</span>.{</span>pollable</span>} </span> </span> </span>/// A timestamp in nanoseconds. </span> </span>type instant </span>= </span>u64 </span> </span> </span>/// Read the current value of the clock. </span> </span>/// </span> </span>/// The clock is monotonic, therefore calling this function repeatedly will </span> </span>/// produce a sequence of non-decreasing values. </span> </span>now</span>: func() -> instant </span> </span> </span>/// Query the resolution of the clock. </span> </span>resolution</span>: func() -> instant </span> </span> </span>/// Create a `pollable` which will resolve once the specified time has been </span> </span>/// reached. </span> </span>subscribe</span>: func( </span> </span>when</span>: instant, </span> </span>absolute</span>: bool </span> ) -> pollable </span>} </span></code></pre> A Wasm Component is a single typed object consisting of a Core WASM Module plus corresponding WIT definitions. Components can specify they either export</em> specific interfaces, or require that other interfaces are imported</em>. For example, I could write a binary program which prints the number of seconds elapsed, which exports</em> a main function with no arguments, and imports</em> both the stdout</code> and monotonic_clock</code> interfaces.</p> One way to think about Wasm Components is: "What if we had ML-style modules in our linker?", and that thought was taken all the way through from the system call layer to the way libraries are linked to each other. With the added benefit that each Wasm Component operates as its own security boundary, using a "shared-nothing" approach to ensure isolation.</p> What is WASI?</h2> WASI stands for the "WebAssembly System Interface". Some people have recently also started dubbing it the "WebAssembly Standard</em> Interfaces", since it covers far more than just operating system interfaces. The way I think about this is as sets of standard</em> Wasm Components which can be implemented by any vendor and targeted by any toolchain. This can include APIs such as socket-based networking, or filesystem access. But also APIs such as HTTP-based clients and servers, or even message queue interfaces.</p> Not all WASI interfaces are created equally though. For example: a serverless environment may not want to expose direct access to the filesystem. Or the stdlib of a programming language may not want to provide access to message queue APIs. This differentiation in goals is why WASI has a notion of "Worlds". The different "Worlds" are still in the process of being defined, but the following two are actively being worked on right now:</p> A world modeling a traditional operating system, providing access to sockets, stdio, filesystem access, etc.</li> A world modeling a "bursty" environment, typically best suited for serverless applications. This will include access to handling HTTP requests, making HTTP requests, access to object storage, message queues, etc.</li> </ul> The way WASI interfaces are standardized is via the W3C's WebAssembly working group. People from across the industry come together to work on defining these interfaces, which once accepted are then publish as a standard. This is "standard" with a capital s. The standardization process can take time, but it has the upside that once ratified you can know for a fact just about everyone in the space will be adopting it.</p> Categorizing WebAssembly</h2> Now that we've talked about what Wasm and WASI are</em>, let's talk about what they aren't</em>. Or well, perhaps more: what they aren't just</em>. WebAssembly falls into multiple categories all at once, which means it also kind of escapes categorization entirely. Depending on which angle you take, you can think of WebAssembly as:</p> A programming language</strong>: The WebAssembly text format is based on S-Expressions</a>, and it can be authored by hand if you want to. WIT is a similar format targeted towards linkers, and incorporates a lot of work from the ML world.</li> An operating system</strong>: Whether you're running WASI on Windows, on Linux, or directly on hardware - the host you're targeting is WASI, and it should behave the same everywhere. From a programmer's perspective, WASI becomes</em> the operating system. The exact details of what's backing the runtime shouldn't matter.</li> A virtual machine</strong>: Wasm provides strict sandboxing guarantees, meaning that you're able to trust a WebAssembly runtime with the same guarantees as you would trust a VM. A Wasm program cannot ever jump out of its current memory, and access memory of a different program running on the same computer. But it goes one step further:</li> An application runtime</strong>: Part of WASI are definitions for service-level APIs such as "HTTP handling" and "message queue access". APIs which wouldn't be out of place in an application runtime such as dotnet, or a cloud vendor such as Azure.</li> A single-host container orchestrator</strong>: rather than dynamically linking applications on the same host via HTTP, WASI provides an alternate model to dynamically link programs without sharing any memory.</li> A compiler</strong>: WebAssembly itself is a bytecode format which is the target of language toolchains. But the Wasm runtimes themselves need to take that bytecode and convert it to machine code, which means there is a lot of compiler work involved</a>.</li> An ABI, memory model, and linker</strong>: Wasm Components in many ways are</em> the platform ABI. And because of how sandboxing has been implemented, one could argue that Wasm has its own memory model. And the linker is basically an ML module system in disguise. All put together, it's basically its own model of a system, which behaves quite differently from most traditional platforms.</li> </ul> In "Embrace the Kinda"</a> SunfishCode talks about the categorical ambiguity of WebAssembly in more detail.</p> Iterating on WASI</h2> The latest version of WASI currently available is called: "WASI Preview 1". For the past four years people have been hard at work on defining "WASI Preview 2", and everything I've written about in this post has been about Preview 2. The Preview 1 version of WASI was much closer to just</em> an operating system layer. But it quickly became clear that this would hit some pretty big limitations, and if WASI wanted to live up to its stated goals, it would need to change.</p> Preview 2 is a complete rework of Preview 1, introducing WIT, WASM Components, and all sorts of new standard interfaces. What it doesn't yet do is provide first-class async primitives, which is scheduled for WASI Preview 3. Threads are also missing from Preview 2, and work on that is still ongoing.</p> One neat thing of the way WASI is structured is that virtualization layers can be nested. In order to make upgrading between WASI versions easier, a shim, a shim will be provided which allows Preview 1 code to continue working in hosts which only implement Preview 2.</p> Preview releases are intentionally backwards-incompatible, and the idea is for them to be deprecated over time. Eventually the plan is to release a "WASI 1.0" specification which will</em> provide more stability. The hope, at least, is that with each WASI preview release, the number of changes between major versions will shrink - so that the final 1.0 release will represent mostly a formalization of what people have already been using for a while.</p> The future of WASI</h2> As mentioned, WASI Preview 2 doesn't yet have a model for first-class async, or for threading. But it also doesn't yet provide any reasoning for multi-host (distributed) linking of programs, nor are all of the WASI interfaces fully specified yet. These are all things coming down the pipeline, which I'm really excited about. Specifically for first-class async, the plan is to model that using a structured model. Having access to that in</p> Beyond core WASI features, there is a lot of other interesting work ongoing. Maybe most interesting in my opinion is the paper: "Going beyond the Limits of SFI: Flexible and Secure Hardware-Assisted In-Process Isolation with HFI"</em></a> which was a joint project between UCSD, Intel, and Fastly to provide a new set of instructions to make in-process sandboxing faster and cheaper. This should greatly benefit WebAssembly, allowing it to sandbox with even less overhead.</p> Work on WASI Preview 2 is currently ongoing, and should be released later this year. On behalf of Microsoft I'm currently working on the Preview 2 implementation for rustc together with folks from Fastly and Fermyon. But work is simultaneously happening for other language toolchains, runtimes, and platforms as well. With WASI Preview 2 including async socket support (though not yet multi-threading), I think this may finally be the year WebAssembly is finally going to start living up to its expectations. And I'm incredibly excited for that!</p> Conclusion</h2> The way I roughly think about WebAssembly is as a hopeful vision of what computing can be. It represents an opportunity to take the last 40 years of operating system research, compiler development, and industry experience and combines that into a form that is both coherent and accessible. I hope this post can provide some insight on what WebAssembly and WASM are, and where things are headed.</p> Appendix A: Terminology Cheat Sheet</h2> Wasm</strong>: short for "WebAssembly".</li> WebAssembly</strong>, or "Core WebAssembly": A bytecode format which can be translated to native code by a "WebAssembly interpreter" or "WebAssembly runtime"</li> WASI</strong>: short for "WebAssembly System Interface". Sometimes also referred to as: "WebAssembly Standard Interface". It provides a set of standardized component interfaces using the WIT IDL language.</li> WIT</strong>: short for "WebAssembly Interface Types" is a language to define inter-component interfaces. It is used as part of WASM Components.</li> IDL</strong>: "Interface Description Language" is a meta-language which defines program interfaces. WIT is an example of an IDL.</li> WASM Module</strong>: best thought of as "object files" for WebAssembly. It's a single binary object containing core WASM bytecode.</li> WebAssembly Component</strong>: a typed wrapper around a WASM Module. It combines a typed WIT interface and WASM module to create a typed component.</li> WebAssembly Worlds</strong>: A combination of different WASI interfaces, packaged up into single environments. The brunt of development work in this space is currently happening to define an "operating system world", and a "bursty world" (serverless).</li> WASI Preview 1</strong>: The snapshot of WASI development which was released in 2019. This wasn't originally versioned and was previously just referred to as "WASI".</li> WASI Preview 2</strong>: The version of WASI which is being releasing later this year, built on top of WASM Components.</li> Linker</strong>: A program which glues a number of other programs together into a single program. The parts which are glued together are usually called "libraries" or "objects", and the output program is usually called a "binary". With WASM Components the differences between libraries and binaries are much less clear.</li> </ul> Pattern Extensions Fri, 26 May 2023 00:00:00 +0000 Microsoft Build is happening this week, and with it come new announcements for the C# language and dotnet runtime. I was watching the video: "What's new in C#12 and beyond"</a>, and the C# team talked about pattern matching and pattern constructors. I've been thinking a bit about patterns in Rust recently, and I wanted to share some of my doodles.</p> Disclaimer time: I'm not on the Rust language team, nor do I hold any direct decision making power. This post is not intended to be a complete or consistent proposal, but rather to share some short ideas which seem neat. I'm not suggesting we should prioritize this work over any other work, nor arguing that this should be a high priority item. The reason I'm writing this is because I like to keep up with programming languages - and C# made some cool announcements this week.</em></p> Shapes of types</h2> For the purpose of this post I want to distinguish between three different shapes of types. The shapes are:</p> List types, or "arrays" in Rust</li> Sum types, or "enums" in Rust</li> Product types, or "structs" in Rust</li> String types, such as "hello"</code></li> Ranges, such as 1..5</code></li> </ul> This is intentionally not a comprehensive list; neither on the side of the shapes, nor on the side of the examples. For example: in Rust we also have string-shaped items; and we also have vectors which are a kind of list. But this is intentionally not a complete list - just enough of a categorization to allow us to work through the remainder of this post.</p> Pattern Initializers</h2> In Rust we have different kinds of initializer syntax available today:</p> let</span> x = [</span>0</span>; </span>12</span>]; </span>// Initialize an array containing twelve zeros </span>let</span> f = Foo {}; </span>// Initialize a struct "Foo" </span>let</span> b = Result::Ok(</span>12</span>); </span>// Initialize the enum `Result` to `Ok` </span>let</span> r = </span>1</span>..</span>5</span>; </span>// Initialize a range of one to five </span>let</span> s = "</span>hello</span>"; </span>// Initialize a static str to "hello" </span></code></pre> Let's take the list-initializer syntax as an example. Say we wanted to initialize not an array, but a vec. The way we would write that today would be by doing:</p> let</span> x = vec![</span>0</span>; </span>12</span>]; </span></code></pre> The vec![]</code> macro is built into Rust, and imported by default so this works for Vec</code>. But if we for example wanted to initialize a VecDeque</code></a> we'd be out of luck for the short-hand syntax, and we'd need to go through a struct constructor instead.</p> In C# 12 a new feature is added which is called "collection literals" (demo</a>, spec</a>) which allows you to opt-in types to make use of constructor literal syntax. In the demo they take an example of ImmutableList</code>, and allow it to be constructed using the same syntax as a regular list. You can watch the demo or read the spec to get a full picture of how this works in C#. But in Rust we could imagine doing something very similar via a trait. Roughly sketching it out, you could imagine this could work something like this:</p> let</span> v = [</span>0</span>; </span>12</span>]; </span>// array </span>let</span> v: Vec<_> = [</span>0</span>; </span>12</span>]; </span>// Vec </span>let</span> v: VecDeque<_> = [</span>0</span>; </span>12</span>]; </span>// VecDeque </span></code></pre> Pattern Matching</h2> Pattern matching is another interesting feature in Rust. In C# pattern matching was added in (if I'm not mistaken) C# 6, and has seen steady improvements over the past number of releases. In Rust we've had pattern matching since the very beginning, but what we can match on has remained fairly limited to a number of built-ins over time.</p> Let's take a look again at the venerable list type in Rust: the array. We can match on arrays by using the []</code> notation. Take for example the following, whose first arm will match on the content of the array because it's the same:</p> let</span> x = [</span>12</span>, </span>13</span>]; </span>match</span> x { </span> [</span>12</span>, </span>13</span>] => {} </span> _ => {} </span>} </span></code></pre> For arrays that's all well and good. But for other list types it's more complicated. Take for example the Vec</code> type: we can't directly match on it using the same []</code> syntax. Instead we must obtain a slice</em> first via the .as_slice</code> method or using the index syntax ([..]</code>):</p> let</span> x = vec![</span>12</span>, </span>13</span>]; </span>match</span> x[..] { </span> [</span>12</span>, </span>13</span>] => {} </span> _ => {} </span>} </span></code></pre> But while we have a workaround to enable matching for vec, we don't have that option for many other container types. For example: VecDeque</code> doesn't necessarily hold items in a single continguous slice of memory, so we can't index into it directly using [..]</code> like we can with vec. The easiest way to do this is to call VecDeque::make_contiguous</code></a> to reorder items in memory and return a single unified slice. This does however come with a runtime cost, which means we're now trading convenience for performance:</p> let mut</span> x = VecDeque::new(); </span>x.</span>push_front</span>(</span>13</span>); </span>x.</span>push_back</span>(</span>12</span>); </span>match</span> x.</span>make_contiguous</span>() { </span>// this will re-allocate </span> [</span>12</span>, </span>13</span>] => {} </span> _ => {} </span>} </span></code></pre> It would be far nicer if we could directly affect the matchers in Rust, rather than needing to work via slices. A lot of interesting container types don't hold memory in contiguous slices, so obtaining a slice can be difficult or expensive. But it is often reasonably cheap to visit items in the container, which is all we would need to enable pattern matching to work on richer types. Just like with initializer patterns, it'd be nice if there was a way to enable pattern matching to be implementable via the trait system. What if this was possible instead?:</p> let mut</span> x = VecDeque::new(); </span>x.</span>push_front</span>(</span>13</span>); </span>x.</span>push_back</span>(</span>12</span>); </span>match</span> x { </span>// no re-allocation because we match directly </span> [</span>12</span>, </span>13</span>] => {} </span> _ => {} </span>} </span></code></pre> Pattern Types</h2> Another interesting potential extension to Rust's pattern system is the notion of "pattern types". In my series on state machines</a> I've covered some of this. But Oli pointed out that most of that actually falls under a feature called "pattern types". This would allow us to not only reason about types as we can today, but also about their refinements 1</a></sup>.</p> ^{1</sup> I hope I'm using this word correctly here lol.</p> </div> Let's say we have a "double" function, which takes a number and doubles it. In order for it to fit in the same number type, the number can't be more than half as big. Because if we double it, we don't want to overflow the number type or use a bigger container. If we hand-coded this today, we'd probably want to write something like this:</p>}fn </span>u32_double</span>(</span>num</span>: </span>u32</span>) -> </span>u32 </span>{ </span> </span>if</span> num >= </span>u32</span>::</span>MAX </span>/ </span>2 </span>{ </span> panic!("</span>number cannot be doubled without overflowing</span>"); </span> } </span> num + num </span>} </span> </span>assert_eq!(</span>u32_double</span>(</span>1</span>), </span>2</span>); </span>// ✅ ok </span>u32_double</span>(</span>u32</span>::</span>MAX</span>); </span>// ❌ panic, numbers are too big </span></code></pre> This has an issue that this check exists entirely at runtime. If we initialize a number literal such as in our passing example, we hope that the compiler's optimizer kicks in and removes the check. But there is no guarantee that it will. Instead using pattern types we can move this guarantee into the type system, and we can omit the check in most cases. Using entirely made-up syntax, you could imagine this could look something like this:</p> // lol, idk what this should look like. let's just pretend ok? </span>const</span> N: </span>u32 </span>= </span>u32</span>::</span>MAX </span>/ </span>2</span>; </span>fn </span>u32_double</span>(</span>num</span>: </span>u32</span> is 0..N) -> </span>u32 </span>{ </span> num + num </span>} </span> </span>assert_eq!(</span>u32_double</span>(</span>1</span>), </span>2</span>); </span>// ✅ ok </span>u32_double</span>(</span>u32</span>::</span>MAX</span>); </span>// ❌ compile error: doesn't fit in the pattern range </span></code></pre> Pattern types would be nice for a number of other reasons too. As previously mentioned, I believe they would make it easier to work with the type state pattern in Rust. But I also believe they would allow to be more specific about borrowing, enable libary authors to define their own NonZero</code> types. But also provide us with what I believe may be the most ergonomic syntax for try</code>-functions:</p> // function notation </span>try </span>fn </span>foo</span>() -> </span>u32</span> throws Err<io::Error> {..} </span>// Result </span>try </span>fn </span>foo</span>() -> </span>u32</span> throws None {..} </span>// Option </span>try </span>fn </span>foo</span>() -> </span>u32</span> throws Break<()> {..} </span>// ControlFlow </span> </span>// closure notation </span>try \|\| {} </span>// infer everything </span>try \|\| throws None {} </span>// infer success path </span>try \|\| -> </span>u32 </span>{} </span>// infer error path </span>try \|\| -> </span>u32</span> throws None {} </span>// infer nothing </span></code></pre> Tangent time: (it's my blog and you can't tell me what to do). But what I like about this in particular is that it syntactically separates the fallibiliy aspects of the function from the base of the function. This is especially nice because for closure notations this allows us to provide partial ascriptions. Most of the time you can probably just write try \|\|</code> and the inference engine will help you out. But it's nice when you can write more of the signature when you need to. Anyway, to close this out: I think pattern types are pretty neat and they feel like they are a bit of a non-feature. In the sense that they (at least to me) don't really change anything about the idea of Rust, they just allow more code to compile which people reasonably could assume should already compile today.</p> the "is" keyword</h2> Rust recently (1.42, I know I know) stabilized the matches!</code> macro which allows you to use patterns for comparisons. The way this works is as follows (taken from the docs):</p> let</span> foo = '</span>f</span>'; </span>assert!(matches!(foo, '</span>A</span>'..='</span>Z</span>' \| '</span>a</span>'..='</span>z</span>')); </span> </span>let</span> bar = Some(</span>4</span>); </span>assert!(matches!(bar, Some(x) </span>if</span> x > </span>2</span>)); </span></code></pre> It takes a variable on the left side, and a pattern on the right side, and then compares the two. This is often nice to have when working within expressions such as macros. Prior to the macro, it was common (and still is common) to add is_</code> methods to enums, such as: Result::is_ok</code></a> to allow easy comparisons within expressions. In C# (and other languages too) there exists the is</code> keyword</a> which elevates this to a language feature. With it we could rewrite the previous example as:</p> let</span> foo = '</span>f</span>'; </span>assert!(foo is '</span>A</span>'..='</span>Z</span>' \| '</span>a</span>'..='</span>z</span>'); </span> </span>let</span> bar = Some(</span>4</span>); </span>assert!(bar is Some(x) </span>if</span> x > </span>2</span>); </span></code></pre> Whether this feature would carry its weight is a good question. Right now it's good practice to add is_</code> methods for each enum member, which is a lot of typing. So much so that I added a generator to Rust-Analyzer for it</a> to automate the process. If every member in every enum would benefit from a corresponding method, maybe having it as a language feature would not be a bad idea? 2</a></sup></p> ^{2</sup> Tangent: I'm still wondering about a different timeline where the as</code> keyword would make use of Into</code> instead of doing primitive casts. And as?</code> could be mapped to TryInto</code>. I don't know enough about the topic to say anything about why that isn't the case / couldnt't be made to work. But conceptually I kind of like the idea of having both as</code> and is</code> as a pair of keywords to operate on types.</p> </div>}On Product Patterns</h2> At the start of this post I mentioned there are a number of shapes types can be, but so far we've only looked at list types. I've been looking at C# and while support for matching on product types (C# Records</a>) seems to roughly match to what Rust is doing - I'm less sure about what the story will be for constructing or matching on e.g. hashmap structures. It seems the following syntax (collection initializers</a>) is supported today to construct hashmaps:</p> var </span>data </span>= new Dictionary<</span>string</span>, </span>string</span>> { </span> { "</span>test</span>", "</span>val</span>" }, </span> { "</span>test2</span>", "</span>val2</span>" } </span>}; </span></code></pre> This isn't quite the same as collection literals</em>, and I also don't know what that the future plans for pattern matching are in C#. Also I'm kind of coming up empty for other languages on how e.g. matching on arbitrary product types should look like. That's a long way of saying: for any record types that aren't just structs, I'm unsure what the syntax should be. But because we're fun-posting, here's three potential options that I'm somewhat certain are all not a good idea:</p> // Ehhhh option 1: reinterpret struct tokens as </span>// values. This seems really weird and bad? </span>// Downside: record names are not string literals, </span>// and string literals are not the only possible keys. </span>let</span> x: HashMap<&</span>'static str</span>, </span>u32</span>> = { </span> hello: </span>12</span>, </span> world: </span>13</span>, </span>}; </span>match</span> x { </span> { hello: </span>12</span>, world: </span>13 </span>} => {} </span> _ => {} </span>}; </span> </span>// Ehhhh option 2: Literals in both the key and the value </span>// position kind of like JS Object notation. </span>// Downside: this syntax is ambiguous, because it's </span>// the same as block scope syntax. </span>let</span> x: HashMap<&</span>'static str</span>, </span>u32</span>> = { </span> "</span>hello</span>": </span>12</span>, </span> "</span>world</span>": </span>13</span>, </span>}; </span>match</span> x { </span> { "</span>hello</span>": </span>12</span>, "</span>world</span>": </span>13 </span>} => {} </span> _ => {} </span>}; </span> </span>// Ehhhh option 3: The only initializer syntax is [], </span>// so we tuple our way through it. </span>// Downside: this implies ordering where there is none. </span>let</span> x: HashMap<&</span>'static str</span>, </span>u32</span>> = [ </span> ("</span>hello</span>", </span>12</span>), </span> ("</span>world</span>", </span>13</span>), </span>]; </span>match</span> x { </span> [("</span>hello</span>", </span>12</span>), ("</span>world</span>", </span>13</span>)] => {} </span> _ => {} </span>}; </span></code></pre> Anyway tldr: where pattern initializers and matchers for list-like types seem like a relatively straighforward design, a syntax for product types (maps, sets) seems much less obvious to me.</p> On String Patterns</h2> In Rust 2021 we already reserved token prefixes</a> with the intent of allowing new string literals such as f"hello {world}"</code> for format strings or s""</code> for allocated strings. So we're already somewhat on the way to expand initializer syntax. I think it would make sense if we could work in the inverse and as we work on that also allow the inverse to work:</p> // Rust today </span>match </span>String::from("</span>hello</span>") { </span> "</span>hello</span>" => {} </span>// ❌ Compile error: expected `String`, found `&str` </span> _ => {} </span>} </span> </span>// What if something like this could work instead? </span>// (string literals + string matching) </span>match</span> s"</span>hello</span>" { </span>// String literal constructor </span> "</span>hello</span>" => {} </span>// String pattern matching (without casting to string slice) </span> _ => {} </span>} </span></code></pre> Conclusion</h2> As I mentioned at the start, this is a bit of a rambly/braindumpy post because I saw some cool C# things today and I wanted to write about that. I think it's fun to think through what it could look like if Rust's pattern system became more capable. To briefly recap what we've shown about pattern initializers + matchers:</p> // Creating + matching on a `VecDeque` today. </span>let mut</span> x = VecDeque::new(); </span>x.</span>push_front</span>(</span>13</span>); </span>x.</span>push_back</span>(</span>12</span>); </span>match</span> x.</span>make_contiguous</span>() { </span> [</span>12</span>, </span>13</span>] => {} </span> _ => {} </span>} </span> </span>// Creating + matching on a `VecDeque` if we had </span>// implementable list initializers + list patterns </span>let mut</span> x: VecDeque = [</span>12</span>, </span>13</span>]; </span>match</span> x { </span> [</span>12</span>, </span>13</span>] => {} </span> _ => {} </span>} </span></code></pre> That's all. ✌️</p> Linear Types One-Pager Tue, 28 Mar 2023 00:00:00 +0000 This post represents an overview of an MVP "linear types" design which we could probably start implementing and validating today</em> if we wanted to. What I'm sharing here is a combination of conversations I've had with Gankra</a> and Jonas Sheevink</a>.</p> Prior Reading</h2> The Pain Of Real Linear Types in Rust</a></li> Must move types</a></li> Linearity and Control</a></li> </ul> What are linear types?</h2> We typically frame linear types as: "Real #[must_use]</code> types"</em>. But it seems like what we should be doing is framing linear types as: "Types which guarantee their destructor will be called."</em> This is enough to provide the benefit we feature we actually want: a guarantee that destructors will be called, so we can rely on them for soundness purposes.</p> A minimal implementation</h2> The estimate is that something like this would probably take a skilled compiler engineer on the order of days, not weeks or months, to validate:</p> We define a new unsafe</code> (auto-)trait named Leak</code>/Leave</code>/Forget</code></li> All bounds take an implicit + Leak</code> bound, like we do for + Sized</code>.</li> Certain functions such as mem::forget</code> will always keep taking + Leak</code> bounds.</li> Functions which want to opt-in to linearity can take + ?Leak</code> bounds.</li> Types which want to opt-in to linearity can implement !Leak</code> or put a PhantomLeak</code> type in a field.</li> </ul> This wouldn't require any special type of Drop</code> either, we just guarantee it will always be called for all !Leak</code> types. I have a feeling this design is actually really close to what Niko was describing in his post. The main difference is that we consider a destructor being run as enough to satisfy the "must use" semantics of linear types. So the only difference between linear and affine types is that linear types can't be safely passed to APIs such as mem::forget</code> and Arc::new</code>.</p> Interactions with control flow</h2> The only way to prevent Drop</code> from running is if we pass it to mem::forget</code>, create a cycle using Rc</code>, Arc</code>, or using static</code>. The first relies on the ManuallyDrop</code> built-in, the second relies on the UnsafeCell</code> built-in, and the last is a language item. We can disallow the use of !Leak</code> types in static</code> lang items. And for the functions we can make it so the bounds will always need to take + Leak</code>, meaning all of their derivatives will too. If a function wants to use take ?Leak</code> as a bound and pass it to + Leak</code> types, the only way to do that is via an unsafe cast. That would mean it is on the hook for upholding the safety invariants of the ?Leak</code> type. All other interactions with e.g. panic!</code>, .await</code>, or ?</code> would Just Work as expected and we don't need to do anything for them.</p> If people want to put a !Leak</code> type in an Arc</code>, they can create an unsafe wrapper which temporarily removes the !Leak</code> bound from the type, and put that in an Arc</code>. They'd then be manually on the hook for ensuring destructors are run, but that's okay.</p> For async types, we don't yet have a async Drop</code> design. But !Leak</code> types would guarantee that async Drop</code> is always run. This would be enough to guarantee the "must consume" semantics we want for certain types.</p> Drawbacks and Challenges</h2> This scheme would work and could be implemented in record time. In my opinion we should do this on nightly, just to prove that it can be done. Once done we can tackle the ergonomics issues this creates:</p> We would want to go through the entire stdlib and mark almost every generic param as + ?Leak</code>.</li> Because trait bounds are instantly stable, we would have no trial period to test out a linear bound, before committing to it.</li> Just like how ?Sized</code> describes "maybe-dyn", ?Leak</code> describes "maybe-must-move". Because we're talking in terms of negation, it is really hard to reason about. Our dyn system is notorious for being some of the hardest to understand part of Rust.</li> Assuming all but a few bounds in the stdlib will eventually take ?Leak</code>, will we ever want to switch to make that the default across an edition to reduce line noise? If we think we might, we should think about what that would look like.</li> Leak</code> introduces new safety rules for Rust which must be upheld when declaring ?Leak</code> or !Leak</code> bounds. Those will need to be spelled out in detail before Leak</code> can be stabilized.</li> </ul> I believe that just like with dyn</code>, "must not move", send, etc we should look at an alternate formulation of these bounds by treating them as built-in effects. That would allow us to address the issues of versioning, visual noise, etc. in a more consistent and ergonomic way. But that's not a requirement to start testing this out, so if we believe we want this feature, starting with the design in this document seems to me like the right way to go.</p> Updates</h2> update 2023-05-03:</strong> Jack Rickard</a> pointed out that !Leak</code> types and statics</a> have bad interactions. Types which are placed in statics don't have their destructors run when the program exits. Which means that if we allowed !Leak</code> types to be placed inside statics, it would prevent destructors from running entirely. This would break the linearity guarantees. so statics have to be a no-no for linear types. This puts linearity right next to const in that statics are fundamentally disallowed.</p> 2023-05-04:</strong> From talking more to Jack Rickard, we created another example of "safe forget" using thread::spawn</code>:</p> fn </span>safe_forget</span><T>(</span>val</span>: T) { </span> thread::spawn(</span>move </span>\|\| { </span> </span>let</span> val = val; </span>// move `val` into the thread </span> </span>loop </span>{ std::thread::park(); } </span>// park the thread indefinitely </span> }); </span>// drop the thread thandle </span>} </span></code></pre> The thread handle becomes unreachable, detaching the thread, and holding the value until the main thread exits - at which point destructors won't run 1</a></sup>. This is contrast to thread::scope</code>, which is tree-structured and wouldn't allow doing this. To me this seems to indicate three things:</p> In order to guarantee destructors are run, it's key to prevent cycles. This holds for both data (Rc</code>/Arc</code>) and computation (thread::spawn</code>). I've got a hunch that structured concurrency is very closely connected to what we're trying to do here.</li> If we are to introduce linearity into Rust, we need to follow the path we've taken with const</code>. One API at the time, validating the linearity guarantees at each stage. This emphasizes the need to have a formalization of the safety rules.</li> Even in this model we need to have a way to express ?Leak Fn</code> to enable closing over !Leak</code> types. This adds further credibility to the idea that we will want some form of surface syntax for !leak fn</code>, to be able to declare free functions which can be passed into linear closure position. This is in addition to the existing !leak async fn</code> use case.</li> </ol> ^{1</sup> It makes me wonder about an alternate timeline where JoinHandle</code> would be !Leak</code> and implement "join on drop" semantics. Had that been the case, then thread::spawn</code> would probably have been okay to close over !Leak</code> types.</p> </div>} Linearity and Control Thu, 23 Mar 2023 00:00:00 +0000 Introduction</h2> Update 2023-03-28</strong>: I've published a follow-up to this post</a> which presents an alternate design, reframing linearity as: "Drop is guaranteed to run". To get a full overview of the design space, read this post to get a sense of the problem domain, and the follow-up post for a concrete solution.</em></p> A week ago Niko published a post on linear types</a>, introducing the idea of "must move" types, which he suggested could be implemented through some form of ?Drop</code> bound. It's far from the first time linear types have come up. Five years ago Gankra also published a post</a> on linear types, explaining what they are and why they're hard to get right.</p> In this post I want to build on these two posts; expanding on what linear types are, why they're useful, how they would interact with Rust, and share a novel effect-based design — which unlike many previous attempts would preserve our ability to implement and use destructors.</p> Before we dive in I want to extend an enormous thanks to Eric Holk, for first positing the idea that linearity obligations might be transferable up the call stack or ownership chain. That has been the key insight on which the majority of the design in this post builds.</em></p> ⚠️ Disclaimer: The syntax in this post is entirely made up, and meant as illustrative only. While it may be tempting to discuss syntax, this post is primarily focused on semantics instead. I am not a member of the language team. I do not speak for the language team. This post does not encode any decisions made by the language team. The point of this post is to exchange ideas, in the open, with peers - because in my opinion that is the best way to go about designing things.️</em></p> What are linear types?</h2> If you've been writing Rust for a little while, you'll be familiar with #[must_use]</code>. Decorating a function with this attribute makes it so the types returned from the functions must be used</em> or else it produces a warning. But #[must_use]</code> is limited. "using" in this context means "doing literally anything with the function output". As long as we do that, #[must_use]</code> is happy. This includes, or example, passing it to mem::forget</code> or Box::leak</code>.</p> must_use_fn</span>(); </span>// ⚠️ "unused return value of `must_use_fn` that must be used" </span>let</span> x = </span>must_use_fn</span>(); </span>// ✅ no warnings or errors </span>let </span>_ = </span>must_use_fn</span>(); </span>// ✅ no warnings or errors </span>mem::forget(</span>must_use_fn</span>()); </span>// ✅ no warnings or errors </span></code></pre> However there are often cases where we would like a more rigorous version of #[must_use]</code>. Say we have a Transaction</code> type. It has a method new</code> to create a transaction, a method commit</code> to finalize the transaction successfully, and a method abort</code> to finalize the transaction unsuccessfully.</p> /// A type representing a transaction. </span>struct </span>Transaction { .. } </span>impl </span>Transaction { </span> </span>/// Create a new instance of `Transaction`. </span> </span>fn </span>new</span>() -> </span>Self </span>{ .. } </span> </span>/// Finalize `Transaction` successfully. </span> </span>fn </span>commit</span>(</span>self</span>) { .. } </span> </span>/// Finalize `Transaction` unsuccessfully. </span> </span>fn </span>abort</span>(</span>self</span>) { .. } </span>} </span></code></pre> What we'd like to express here is that if you call new</code>, you must</em> call either commit</code>, or abort</code>. But Rust's type system doesn't let us express this yet. Even Drop</code> wouldn't help us here, since the type system doesn't guarantee it'll ever be called. What we'd like is for the following to be possible:</p> let </span>_ = Transaction::new(); </span>// ❌ "unused return value of `Transaction::new` that must be used" </span></code></pre> The reason for this is that Rust type system supports affine</em> types, but does not support linear</em> types. This is a common misconception, but the difference between them is actually pretty important:</p> affine types</strong> are types which must can be used at most once</em>.</li> linear types</strong> are types which must be used exactly once</em>.</li> </ul> This means that Rust can guarantee you won't use the same type twice (e.g. we systemically prevent double free</a>s from happening); but for example we can't guarantee that values will</em> have methods called on them (e.g. we can't prevent memory from leaking). Being able to express linearity in Rust would likely also enable us to solve some of the other challenges we've been facing:</p> async scopes</strong>: thread::scope</code></a> allows us to share variables between threads without using locks. An async/.await</code> version of this would require linear types to function 1</a></sup>.</li> true session types</strong>: This is best explained as: "What if we could model the semantics of entire network protocols using the type system." We can get close today, but to fully encode session types we require access to linear types [^session-type-limits] 2</a></sup>.</li> more efficient completion-based IO</strong>: APIs such as io_uring</code> have very specific rules about how objects should be passed between the kernel and the program. Linear types should make it possible to model these interactions with significantly less runtime overhead 3</a></sup>.</li> more efficient async C++ FFI</strong>: The Future</code>-equivalent in C++ must</em> be polled to completion because if you don't you trigger UB. This is unlike Rust's Future</code> type which can be dropped to be cancelled. Linear types would allow us to model this behavior directly in Rust's type system.</li> </ul> ^1</sup>Tmandry wrote a good post about the challenges of async scopes</a>. While he doesn't directly spell out how linear types would help solve some of these issues Niko does</a>.</p> </div> ^4</sup>The paper "Session types for Rust"</a> writes in their conclusion: "We have demonstrated that session types can be implemented directly in a language supporting affine types, and argued for its safety. Like linear types, affine types prevent aliasing, but, unlike linear types, fail to promise progress."</em></p> </div> ^2</sup>There's a cool paper</a> and talk</a> titled "Propositions as Types" by Phil Wadler who makes the point that every good idea is discovered</em>, not invented. In the talk at the 37:00 mark he mentions that linear logic and session types seem to correspond to each other, and may in fact provide us with a solution to way to formally encode concurrency and distribution. Whether that's actually true, I don't know for sure - I'm not a professor in theoretical computer science like Phil is, nor have I ever used linear types or session types to say anything from experience. But it certainly seems plausible, which I think should be enough to get anyone excited who's trying to build reliable networked services.</p> </div> ^3</sup>Tbh I'm not an expert on io_uring</code>, so I might not be 100% on point here. I'm basing my understanding of this mostly on the fact that the rio</code></a> library would be considered sound if the destructors could be guaranteed to run. The point of linear types is to provide such a guarantee, which means linearity could be a way to model completion-based IO safely and ergonomically, directly from Rust.</p> </div> To provide a quick example of something which is inexpressible in Rust today: we can't create an async/.await</code> version of thread::scope</code></a> in Rust today. But using linear types, we could require that the future returned by async_scope</code> is awaited, and that in the event of a cancellation, the destructors of async_scope</code> are guaranteed to run for as long as tasks are live, ensuring we maintain Rust's property of preventing all data races in safe Rust 5</a></sup>.</p> ^5</sup>This is a very similar problem Gankra described in here 2015 post on why why mem::forget</code> should be safe</a>. It caused us to remove thread::scope</code> from the stdlib until its reintroduction in 2022. Unfortunately the tricks we applied to make this work in stable Rust today won't work for async Rust; which means we have to solve this issue for real this time</em> if we want it to be possible.</p> </div> let mut</span> a = vec![</span>1</span>, </span>2</span>, </span>3</span>]; </span>let mut</span> x = </span>0</span>; </span> </span>async_scope</span>(\|</span>s</span>\| { </span> s.</span>spawn</span>(\|\| { </span> println!("</span>hello from the first scoped task</span>"); </span> </span>// We can borrow `a` here. </span> dbg!(&a); </span> }); </span> s.</span>spawn</span>(\|\| { </span> println!("</span>hello from the second scoped task</span>"); </span> </span>// We can even mutably borrow `x` here, </span> </span>// because no other tasks are using it. </span> x += a[</span>0</span>] + a[</span>2</span>]; </span> }); </span> println!("</span>hello from the main task</span>"); </span>}).await; </span> </span>// After the scope, we can modify and access our variables again: </span>a.</span>push</span>(</span>4</span>); </span>assert_eq!(x, a.</span>len</span>()); </span></code></pre> Create and consume</h2> Okay, time to get into the details of the design. At the heart of a linear type's lifecycle are two phases: create (or "construct") and consume (or "destruct"). A linear type promises at the type system level that once it's created, it will always</em> be consumed. The way we're modeling this in our design is using type instantiation to create and pattern destructuring</a> to consume</p> ^{6</sup></div>}^6</sup>This example was inspired by Firstyear's post on Transactional Operations in Rust</a>.</p> </div> let</span> txn = Transaction { .. }; </span>// 1. Create a new instance of `Transaction`. </span>let</span> Transaction { .. } = txn; </span>// 2. Consume an instance of `Transaction`. </span></code></pre> No matter what happens in the code, every linear type which is created will always have a matching consume. This means that as long as your program has an opportunity to run to completion it is guaranteed</em> to be consumed. Yay for type systems!</p> In this post we'll be modeling this via the introduction of a new effect kind (or "modifier keyword" if you prefer): linear</code>. Don't worry too much about the name, I'm mostly using it for clarity throughout this post. This effect can be applied to either types or functions; but we'll start with types.</p> Linear types</h3> When a type is marked as linear</code> it means that once a type has been created, in order for it to type-check it must also be consumed. This will be automatically done by the type checker for you, who will gently tell you if you messed up anywhere. But by using unsafe</code> you can also do this by hand if you need to. More on that later though.</p> linear </span>struct </span>Transaction { .. } </span>linear </span>enum </span>Foo { .. } </span></code></pre> This is not the end of linear types though: a key feature of types is that they can be composed into new types. When a type takes a linear type as an argument, the "linearity" obligation transfers to the enclosing type. This means that a linear type can only be held by another linear type:</p> linear </span>struct </span>TransactionWrapper { </span> </span>txn</span>: Transaction, </span>// linear type </span>} </span></code></pre> Linear functions</h3> The second place where the linear</code> notation can be used is on functions and methods. The way this would look is like this:</p> impl </span>Transaction { </span> </span>fn </span>new</span>() -> </span>Self </span>{ .. } </span> linear </span>fn </span>commit</span>(linear </span>self</span>) { .. } </span> linear </span>fn </span>abort</span>(linear </span>self</span>) { .. } </span>} </span></code></pre> Functions which don't take any linear types as arguments don't have to be marked as linear fn</code> which is why fn new</code> is not marked linear here 7</a></sup>. But if a function wants to take a linear type as an argument, it needs to be marked as a linear fn</code>. Every linear function needs to promise it will do one of three things for every instance of a linear type which is passed to it:</p> ^{7</sup> Folks in review asked me: "why doesn't fn new</code> need to be linear?" The real reason is because it doesn't follow the rules I'm setting out in this post. But it could very well be that it would me a lot more clear if only linear functions could yield linear types, in which case this should be linear fn new</code>. As I've said at the start of this post: syntax is something we can dig into once we know the semantics we want to expose. And answering this question really is just a matter of syntax.</p> </div>} It will destruct the instance.</li> It will yield the instance to the caller along with control over execution.</li> It will call another linear function and pass the instance along as an argument.</li> </ol> This might sound easy enough, but in practice this has implications for what can't</em> be done, such as:</p> It will not mem::forget</code> any instances of linear types.</li> It will not create any cycles around instances of linear types using Arc/Rc</code> or similar.</li> </ol> If we can imagine a parallel version of the stdlib where all types and functions which can be linear are</em> in fact linear: mem::forget</code>, Arc::new</code>, and Rc::new</code> would not exist. And our safety rules would disallow creating new linear fn</code>s which don't uphold those rules. Being able to mark which functions uphold the rules of linearity is exactly the purpose of the linear fn</code> notation.</p> Linear contexts</h3> The "must consume" obligation of a linear function only applies to arguments which themselves are also linear. It should be possible to pass non-linear arguments to linear functions without any problem, allowing you to mix and match linear and non-linear types within a linear function.</p> Similarly: it should be possible to create linear types and call linear functions from non-linear functions. This means you should be able to create and consume linear types from your existing functions largely without problems. The exceptions here are probably async</code> contexts, and things like closure captures: both of those create types which wrap existing types. So when they capture a linear type, they themselves must become linear too. We'll get into more detail on how that works later on in this post.</p> This means that it should be possible to for example have an fn main</code> which is not marked linear</code> which creates and consumes linear types in its function body:</p> fn </span>main</span>() { </span>// 1. Non-linear function context. </span> </span>let</span> txn = Transaction { .. }; </span>// 2. Create a new instance of `Transaction`. </span> </span>let</span> Transaction { .. } = txn; </span>// 3. Consume an instance of `Transaction`. </span>} </span></code></pre> If for whatever reason we do find a good reason why linear types should only be accessible from linear contexts, we could do that. Just like with const {}</code> we could create a linear {}</code> context which can be used to close out the linearity effect. But I don't think that is necessary, and I suspect linear types would be easier to use if we didn't require this. This is almost a question of syntax though, so it's probably not worth going too deep into this right now.</p> Linear methods</h3> I've sort of hinted at this, but not said it out loud: for linear types to be really useful it's not enough if we have types which cannot be dropped. What we really want is pairs of methods which need to be called.</p> In Rust we don't have "class constructors" built into the language as a feature. Instead what we have is struct initialization, and methods such as MyStruct::new</code> or MyStruct::open</code> to construct the type. The reason why these work is because they're the only places in the code which have access to the struct's private fields. So to create the struct, they're the only way to do it.</p> impl </span>Transaction { </span> </span>pub fn </span>new</span>() -> </span>Self </span>{ </span> </span>Self </span>{ field1, field2 } </span> } </span>} </span></code></pre> Similarly with linear types what we don't want is for people to require access to private fields to destruct their types. Instead we can just create methods which can do the destruction for us. This means that the only way to close out the linear</code> effect's requirements is to call a destructor method on the type:</p> impl </span>Transaction { </span> </span>pub</span> linear </span>fn </span>commit</span>(</span>self</span>) { </span> </span>let Self </span>{ field1, field2 } = </span>self</span>; </span> } </span> </span>pub</span> linear </span>fn </span>abort</span>(</span>self</span>) { </span> </span>let Self </span>{ field1, field2 } = </span>self</span>; </span> } </span>} </span></code></pre> As we discussed earlier: the only valid thing to do with an instance of a linear type is to either consume it, return it, or pass it to another linear function. "linear destructors" do the latter: they're linear functions which consume a linear type. So from a compiler's perspective this is exactly right.</p> Linear drop</h3> A key feature feature in Rust is the ability to define Drop</code> destructors (RAII</a>). For example when you create a file using File::open</code>, its Drop</code> destructor will make sure that the file is closed when it goes out of scope. Syntactically this makes code a lot nicer to read, because rather than needing to remember which function to call inside a function body, the type implementing Drop</code> will just auto-cleanup all resources for you.</p> To ensure "linear Rust" is as ergonomic as non-linear Rust, code should largely feel the same as well. This includes having access to Drop</code>. The way we could do this is by introducing a linear version of Drop</code>, which rather than take &mut self</code> takes self</code>. And when called requires the existing rules of linearity are upheld. The way this destructor would work is that it would only</em> run when a type goes out of scope. It would not</em> run if a type is manually destructed.</p> People may be wondering at this point what the difference between linear types and affine types then is, if both get access to destructors. With affine types we can't guarantee that destructors will run, so we can't build on it for the purpose of soundness. With linear types we can make that guarantee, so we destructors can be relied on for the purpose of soundness.</p> In our Transaction</code> example, we can imagine that we might to abort the transaction unsuccessfully if it goes out of scope. But we would want to successfully commit it by hand if we've succeeded. We could do that by adding the following linear Drop</code> implementation:</p> impl </span>linear Drop for Transaction { </span> </span>fn </span>drop</span>(linear </span>self</span>) { </span> </span>self</span>.</span>abort</span>(</span>self</span>); </span> } </span>} </span></code></pre> This would make it so if the transaction goes out of scope or if we call abort</code> it aborts. But only if we call commit</code> manually does it complete successfully:</p> fn </span>do_thing</span>() -> Result<()> { </span>// 1. No linear arguments, so no `linear fn` needed. </span> </span>let</span> txn = Transaction::new(); </span>// 2. Create a new transaction. </span> </span>some_action</span>()?; </span>// 3. On error, drop `txn` and call the destructor. </span> txn.</span>commit</span>(); </span>// 4. On success, call `txn.commit`. </span> Ok(()) </span>// 5. Return from the function; no linear types are live. </span>} </span></code></pre> Admittedly I'm not entirely sure whether linear Drop</code> in fact requires taking linear Self</code> to be destructed, or whether it would suffice for Drop to keep taking &mut self</code> as an argument, assuming that if the destructor is called, it will be destructed anyway. The most important difference between linear Drop</code> and regular Drop is that linear Drop will not run if the type is manually destructed. And that unlike regular Drop, linear types guarantee</em> they will be destructed. The exact details of the interfaces are secondary to the semantics we're trying to encode.</p> Control</h2> In this section I'll be covering the interactions of linear types with control-flow primitives such as panic</code> and async</code>. Creating a coherent model for their interactions is a requirement to make linear types practical.</p> Control Points</h3> You may have heard of the term "control flow" to refer to things like branches, loops, and function calls. It describes the "flow" of control over execution. Perhaps you've also heard of "control flow operators" to refer to things like if</code>, while</code>, and for..in</code> which are the specific keywords we use to encode control flow operations. In the compiler all of these things are tracked in a data structure called the "control-flow graph" (CFG).</p> I've found there is a term here missing, which is something I've started calling "control points": places in the code where the callee hands control over execution back to its caller. The caller then may or may not hand control back to the callee 8</a></sup>. Not all control-flow operations are equal though. We can divide them into two categories:</p> ^{8</sup> This does not include points at which control over execution is handed off to another function by calling it. In structured programming languages such as Rust we can treat those as black boxes, so from the function's perspective we only have to observe what happens when they hand control back to us again.</p> </div>} Breaking operations</strong>: hand control back to the caller, and that's it. Examples: return</code>, break</code>, continue</code>, ?</code>, panic!</code>.</li> Suspending operations</strong>: hand control back to the caller, but the caller can choose to hand control back</em> to the callee again later. Examples: .await</code>, yield</code>.</li> </ul> I'm using "caller" and "callee" somewhat loosely here, since Rust is expression-oriented and with things like try {}</code> blocks and inline loop {}</code>, control flow doesn't necessarily need to be handed to a different function. It can be enough to be handed to an enclosing scope. Let's take a look at a reasonably representative Rust function which does some async IO and then some parsing:</p> async </span>fn </span>do_something</span>(</span>path</span>: PathBuf) -> io::Result<Table> { </span> </span>let</span> file = fs::open(&path).await?; </span> </span>let</span> table = </span>parse_table</span>(file)?; </span> Ok(table) </span>} </span></code></pre> And here it is again, with all control points fully annotated:</p> async </span>fn </span>do_something</span>(</span>path</span>: PathBuf) -> io::Result<Table> { </span> </span>// 1. futures start suspended, and may not resume again </span> </span>let</span> file = fs::open(&path).await?; </span>// 2. `.await` suspends, and may not resume again </span> </span>// 3. `?` propagates errors to the caller </span> </span>// 4. `fs::open` may panic and unwind </span> </span>let</span> table = </span>parse_table</span>(file)?; </span>// 5. `?` propagates errors to the caller </span> </span>// 6. `parse_table` may panic and unwind </span> Ok(table) </span>// 7. return a value to the caller </span>} </span></code></pre> While many of the control points in this function are visible, many others are not. In order for linear types to function, they need to account for all</em> control points regardless of whether they're visible or not. This is why it's so important to be able to implement linear Drop</code> destructors. Because without them it wouldn't be possible to carry linear types over control points.</p> Liveness</h3> Now that we've covered what control points are, we can discuss how they interact with linear types. In its most basic form: when a linear type is held live across</em> a control point, the compiler cannot prove that it will be consumed. Assuming we take our earliest version of Transaction</code> which didn't implement linear Drop</code>. If you hold a an instance of it across an .await</code> the compiler should throw an error. The ?</code> may cause the function to return, meaning there is no guarantee txn.commit</code> will ever be invoked:</p> fn </span>do_something</span>() { </span> </span>let</span> txn = Transaction::new(); </span>// 1. Create an instance of `Transaction` (assuming no `linear Drop`). </span> </span>some_action</span>()?; </span>// 2. ❌ `?` may cause the function to return with an error. </span> txn.</span>commit</span>(); </span>// 3. ❌ we cannot guarantee `txn.commit` will be called. </span>} </span></code></pre> This is the reason why in order for linear types to be practical, they must</em> implement a form of "consuming Drop". Without it we couldn't hold any linear types across control points. And because every function in Rust is currently allowed to panic, it means having linear types without destructors would be entirely impractical</strong>. That's why its likely virtually every linear type will want to implement some form of linear Drop</code>, and there are open questions on how deeply we could integrate this into the language:</p> fn </span>do_something</span>() { </span> </span>let</span> txn = Transaction::new(); </span>// 1. Create an instance of `Transaction` (assuming `linear Drop`). </span> </span>some_action</span>()?; </span>// 2. ✅ `?` may cause the function to return with an error, invoking `txn.drop`. </span> txn.</span>commit</span>(); </span>// 3. ✅ Or `?` does not return, and `txn.commit` is called. </span>} </span></code></pre> The key interaction between linear types and control points is one of liveness</em>. When interacting with a control point, linear types must guarantee they will remain live</em>. This means the linear type must either being returned from a scope via the control point, or must be able to be destructed before the control point finishes handing control back over (e.g. linear Drop</code>). I hope the terminology of "liveness" and "control points" can provide us with the missing vocabulary to talk about how linear types interact with things such as panics and errors.</p> Abort points</h3> It's important to distinguish between "control points" and what I'm calling here: "abort points". Where control points represent points in the control-flow graph where control over execution is handed off to another place within the program. "Abort points" represent points within the program at which point the program stops executing entirely.</p> In practice, every line of code represents an abort point. There is no guarantee a computer will execute the next instruction on a CPU. The best it can do is guarantee that if</em> a next instruction will be run, which one it will be. Power plugs can be pulled, CPUs can shut down processes, kernels can crash. These are things which live outside the type system, and we cannot model within the type system. The type system assumes that the hardware it runs on is well-behaved. If it isn't, the model doesn't hold.</p> When we write panic=abort</code> in our Cargo.toml</code> it changes the meaning of panic!</code> from "unwind" to "just immediately stop". Similarly: we can use std::process::exit</code></a> to stop a program then and there. Because we exit the confines of the program, the type system cannot meaningfully say anything about it, so linear types can't model it.</p> Similarly: we can't treat slow</em> operations as control points either. If a program runs thread::sleep(10_years)</code>, as far as the type system is concerned that doesn't represent a control point anymore than thread::sleep(10_nanos)</code> would. The only time we can really capture that a linear type is violated is in the case of unreachable</em> code. Take for example the following example:</p> fn </span>do_something</span>() { </span> </span>let</span> txn = Transaction::new(); </span>// 1. Create an instance of `Transaction`. </span> </span>loop </span>{} </span>// 2. Loop infinitely </span> txn.</span>commit</span>(); </span>// 3. Complete the transaction. </span>} </span></code></pre> … which should probably produce an error like this:</p> error: linear type destructor is unreachable </span> --> src/lib.rs:4:1 </span> \| </span>3 \| loop {} </span> \| ------- any code following this expression is unreachable </span>4 \| txn.commit(); </span> \| ^^^^^^^^^^^^^ unreachable destructor </span> \| </span></code></pre> Async and Linearity</h2> Async Rust provides us with the ability to write non-blocking code in a imperative fashion, and through its design unlocks two key capabilities:</p> The ability to execute any number of async operations concurrently</em>.</li> The ability to cancel</em> any operation before it completes by dropping its corresponding future.</li> </ol> "async", much like "linearity" can best be thought of as an effect</em>. And when we talk about combining async an linearity, what we're really discussing is effect composition</em>. Effects are built into the language, and hold a deep semantic relationship to the underlying program. This means that when they interact we get to choose how they should behave together. In the case of composing linear and async, there are two behaviors we want to be able to encode:</p> must-await futures</strong>: these are futures which must be awaited, but once awaited may still be cancelled. Because cancellation is a very useful property for most kinds of futures.</li> must-consume futures</strong>: these are futures which not only must be awaited, even if attempted to be cancelled, they must continue doing their work.</li> </ol> In order for a future to be .await</code>ed</a>, it must first be "stack-pinned"</a>, resulting in a Pin<&mut T></code> type. In the general case this means we've permanently lost access to the inner T</code> type: once a type has been pinned, it must remain pinned until it's dropped. This means that once we start polling a future, we can no longer destruct it. That's why for async Rust we need to expand the destructor rules of linearity slightly. In order to "consume" a linear future, it should be enough to place it inside a .await</code> expression. Awaiting a future should be considered equivalent to consuming a future, even if we cancel it later on. This provides us with "must-use" futures:</p> let </span>_ = </span>meow</span>(); </span>// ❌ "`meow` returns a `Future` which must be `.await`ed" </span></code></pre> In order to encode "must-consume" futures we will need access to an async version of linear drop. As Sabrina Jewson</a> pointed out before: if we require a future to continue doing work when cancelled, it can continue doing that work in its drop handler. If the drop handler itself is un-cancellable, then we are able to construct un-cancellable futures.</p> Unlike the linear Drop</code> we've shown before, async linear Drop</code> cannot take self</code> by-value because of the pinning rules. It instead it can only really take &mut self</code>. So again, the fact that it's being destructed will need to be guaranteed by the fact that it's .await</code>ed. Encoding the linearity of async linear Drop</code> itself is a different matter. And I suspect we'll need to dedicate a separate post to exactly spell out how that should work. My guess is that this will require a connection that is as deep to the language as non-async Drop</code>. And that at the root of the program it will bottom out in something like it being a linearity violation if you don't handle the future returned by Drop</code> to completion. I'm not exactly sure yet how this should be done, but it seems like it should be doable.</p> Cycles and Safety</h2> Unsafe Rust provides access to raw pointers. These are pointers, which unlike references, may alias if you're not careful. But just because they could</em> alias doesn't mean they're allowed to alias. And it's in fact undefined behavior if you fail to uphold Rust's safety rules in unsafe Rust. unsafe</code> only removes the guard rails, it doesn't change the rules.</p> The same would apply to linear fn</code>/linear T</code> and unsafe</code>. It should be possible to cast a linear T</code> to a regular T</code>, if you make sure to keep treating T</code> as if it was a linear T</code>. This would allow you to create linear functions which internally could hold cycles, but which externally would not be observable. Assuming we had a "make this type non-linear" (UnsafeNonLinear</code>) equivalent to UnsafeCell</code></a>. We could imagine we could take an instance of Transaction</code> and place it inside an Arc</code> somewhat like this:</p> let</span> txn = Transaction::new(); </span>// create a new linear transaction </span>let</span> txn = </span>unsafe </span>{ UnsafeNonLinear::new(txn) }; </span>// cast it to remove its linearity </span>let</span> txn = Arc::new(txn); </span>// place it inside an `Arc`, may* lead to cycles </span>let</span> txn = Arc::into_inner().</span>unwrap</span>(); </span>// extract it from the `Arc` </span>let</span> txn = </span>unsafe </span>{ txn.</span>into_inner</span>() }; </span>// convert it back to a linear type </span></code></pre> I don't really know whether it makes most sense to create an unsafe wrapper type, or whether we could use as</code>-casts to peel the linearity effect. I would expect casting a T</code> -> linear T</code> would probably be safe, but I don't know for sure. Because this would be a new language construct we would be able to define all of these rules. They seem like they should be fairly straightforward, so fingers crossed it actually ends up being that way.</p> Finally there's an open question we have is whether it could in fact be possible to create linear versions of Arc</code> and Rc</code> (and Mutex</code>/RefCell</code>). If every cloned instance had to prove it was consumed, would it be possible to guarantee that no cycles may occur? Even if that's not possible for the existing APIs, perhaps a different formulation could provide those guarantees. Something like a scope</code> API might be able to provide such guarantees. If this were possible it would make linear APIs significantly more convenient to use, so I suspect it would be worth examining this further.</p> Closure captures</h2> An interesting challenge Gankra shared on Twitter</a> is how closure captures interact with linearity. In order for "linear Rust" to be as ergonomic as non-linear Rust, the two should feel largely similar when used. This means closures and closure captures should be accessible from linear Rust too.</p> Let's examine an example more closely. Here is the Option::map</code> function as implemented in the standard library today. It takes an Option</code>, and a closure, and if Some</code> applies the closure to it. If we take the None</code> branch, at first glance it seems like nothing will happen. But on closer inspection we may realize that we drop the closure, and so we also drop all variables captured by the closure:</p> impl</span><T> Option<T> { </span> </span>pub fn </span>map</span><U>(</span>self</span>, </span>op</span>: impl FnOnce(</span>T</span>) -> U) -> Option<U> { </span> </span>match </span>self </span>{ </span> Some(t) => Some(</span>op</span>(t)), </span> None => None, </span> } </span> } </span>} </span></code></pre> In Niko's blog post, this example was adapted to use T: ?Drop</code>. This is valid because in the Some</code> case the T</code> parameter is passed to the closure, which can consume the linear type. While in the None</code> case there is no T</code>, so there is no type to consume. But as we've mentioned this does not account for the variables captured by the closure, which in this formulation would likely just be disallowed from performing any captures. Instead, using the linear</code> effect we can model this the way we would expect because we have access to linear destructors, and because a linear type can only be captured by another linear type:</p> impl</span><linear T> Option<T> { </span> </span>pub</span> linear </span>fn </span>map</span><U>(</span>self</span>, </span>op</span>: impl linear FnOnce(</span>T</span>) -> U) -> Option<U> { </span> </span>match </span>self </span>{ </span> Some(t) => Some(</span>op</span>(t)), </span> None => None, </span> } </span> } </span>} </span></code></pre> Because T</code> is linear, the closure FnOnce</code> which takes T</code> as an argument must also be linear, and in turn the map</code> method must be linear as well. This does assume that linear closures can be dropped to run the destructors of the variables they've captured. Which seems reasonable, since the Fn</code>-family of traits are marked #[fundamental]</code> and are part of the language - so we can define how they should function in the presence of linear</code>.</p> Linear stdlib</h2> Our linear fn map</code> example doesn't quite capture the same semantics as Niko's proposal yet. This syntax requires</em> types to be linear, whereas in Niko's version types may</em> be linear - or they may not be. In order to capture this "maybe-ness" of effect keywords we need a way to express it: which is what the Keyword Generics Initiative has been working on</a>.</p> To make our effect-based design of Option::map</code> fully compatible with the semantics Niko laid out, we'd have to make one more change. Rather than using T: ?Drop</code> which communicates the presence of an effect through the absence of an auto-trait, we can instead use ?linear T</code> to more directly communicate the potential presence of an effect:</p> impl</span><</span>?</span>linear T> Option<T> { </span> </span>pub </span>?linear </span>fn </span>map</span><U>(</span>self</span>, </span>op</span>: impl ?linear FnOnce(</span>T</span>) -> U) -> Option<U> { </span> </span>match </span>self </span>{ </span> Some(t) => Some(</span>op</span>(t)), </span> None => None, </span> } </span> } </span>} </span></code></pre> From the feedback we've received I know many people are not particularly thrilled about this syntax - which is perfectly understandable. But exact syntax aside, it's important to observe that if we do this right, then this syntax should be mostly temporary. Aside from a few exceptions in the stdlib (Arc</code>, Rc</code>, mem::forget</code>), most library code can probably be marked as linear</code>. Which begs the question: if most code can be linear, could it make sense to eventually assume linearity, unless people opt-out? If that were the case, then the signature of Option::map</code> would remain identical to what it is today in the stdlib already. But the signature of mem::forget</code> could then become:</p> pub const fn </span>forget</span><T: </span>!</span>linear>(t: T) { .. } </span></code></pre> It seems far more desireable to gradually linear</code>-ify the stdlib rather than defining a parallel linear</code> stdlib checked into crates.io</code> (as we've done once already</a> for async</code>). And far more empathic than trying to argue that users of Rust are in fact wrong for wanting to make use of APIs such as Option::map</code> in linear contexts. Regardless of which set of effects you're using in Rust to program with, familiar patterns and techniques should be accessible to all.</p> Movability</h2> In his post Niko describes "linear types" as "must-move types". This begs the question: if there are types which must</em> move, are other kinds of moves possible as well? And there are. By default types in Rust can be described as "may-move types", because they can be moved but they don't have to. And then we also have access to "must-not-move types" which are represented by combining the Pin</code></a> type with the !Unpin</code></a> trait. Though in practice most people prefer to encode the immovability of types using the more declarative pin-project</code> crate</a> instead.</p> must move</strong>: linear types</li> may move</strong>: the default mode of Rust types</li> must not move</strong>: pinned types</li> </ul> Thinking of "must-move types" (linear) as the opposite of "must-not-move types" (pinned) is a useful model to carry forward, as they encode opposing guarantees about the movability of types. Reconciling these opposing guarantees is what makes combining async + linear types difficult, since once futures start being polled can generally no longer be moved. If this post continues into a series, this would be a topic I'd like to revisit in more detail later on.</p> Further Reading</h2> Gankra wrote an excellent post in 2015 on why mem::forget</code> should be safe</a>, explaining why we can't assume destructors will always run in Rust. She wrote a follow-up post in 2017 about what makes linear types hard to model</a>, concluding on an optimistic note with: "Ok having actually written this all out, it’s better than I thought."</p> The Wikipedia page on linear types</a> provides a useful overview of the difference between among other "linear" and "affine" types. Including type systems with both more and less guarantees than both. Phil Wadler's "linear types can change the world!"</a>, provides a great type-theoretical introduction to what linear types are, and why they are useful. Similarly on type-theory side, Phil Wadler also published "Propositions as Types" (paper</a>, talk</a>), making an excellent case that properties such as lambda calculus and linear types are not invented - they are discovered. Meaning they may in fact be a fundamental part of logic and computing.</p> More recently Tyler Mandry wrote a blog post on the challenges of defining scoped tasks in Rust</a>. And Niko Matsakis wrote a post on "must move" types</a>, as a potential way to overcome the challenges scoped tasks present. Miguel Young de la Sota gave a talk about move constructors in Rust</a>, explaining how like in C++, Rust could implement self-referential types which can be moved. Yosh Wuyts (hello) wrote a post</a> on the challenges we face in order to make common interactions with pinned values safe. Sabrina Jewson wrote a post describing the challenges of async destructors</a>, going into great detail about various considerations in the design space. And Yosh Wuyts (hello, again) wrote a post describing how async cancellation</a> works, and why it's an essential capability of async Rust.</p> Conclusion</h2> Linearity in Rust is largely inexpressible today. This leads us to work around this restriction in various ways, which come at the cost of correctness, performance, and ergonomics. In this post we've shown how linearity in Rust could be modeled as an effect, bypassing any of these restrictions. We've shown how it could be used to create ergonomics which are largely similar to existing Rust, and how we could gradually introduce linearity into the stdlib and ecosystem.</p> We've shown how for the sake of ergonomics it's not only important to be able to create and consume bare types, but also to be able to define constructors and destructors which must be called as pairs. Another contribution this post has made is formulated a model for linear Drop</code>; which is required to preserve Rust's existing ergonomics in a linear context.</p> Finally we've taken a closer look at how linearity interacts with cycles, with unsafe</code>, and with async</code>. For async</code> specifically, we've taken a look at how linear async fn</code> would work, why it should not disallow cancellation by default, and how we could use it to still encode "must-not cancel" semantics.</p> To close this post out: I don't expect this post to function as a definitive design. In fact, it was written with the opposite intent. I hope this post will help us better understand the challenges linear types face in Rust, and serve as a useful tool in getting us closer to a design which we would be happy to adopt.</p> I'd like to thank Sy Brand, Iryna Shestak, and Eric Holk for reviewing this post prior to publishing.</em></p> Using HTML as a compile target Fri, 03 Feb 2023 00:00:00 +0000 If you've been around an industry for long enough you end up with Opinions™ about things. Here's one of mine: I think one of the biggest missing aspects of web programming is the lack of support for treating HTML as a compile-target.</p> Most folks reading this blog may not be aware of this, but I started my career working with web technologies, which I kept at for the better part of a decade. I co-authored one of the (at least to my knowledge) first 1</a></sup> CSS-in-JS engines</a>, I've written frontend frameworks</a>, designed web compilers</a>, and authored entire HTTP stacks</a>. Even if web tech isn't really my current area of focus, it's something I've worked on a fair bit, and still care about a lot. So here's a post about an opinion I have, in a field I don't do much in currently, but still think would like to share.</p> ^1</sup>To my knowledge at the time at least. I'll totally buy that e.g. PHP or Google's web compiler may have supported similar stuff - but we didn't know about it at the time. When we designed sheetify in ~2015 ish css-modules</code> was all the rage, and we figured we could do something similar without needing to overload the meaning of require</code>. And we didn't know of anything that worked similar at the time. Later, styled-components</code></a> became a thing, citing our work on sheetify as an inspiration. But that's all ancient history at this point.</p> </div> A taste of HTML</h2> There are well over 100 unique HTML elements in the spec. Let's zoom in on just one of those: the <li></code> element</a>. It inherits from a class hierarchy which is 4 levels deep</a>. It exposes two unique attribute values, of which 1 has been deprecated and shouldn't be used. And it can't just be placed anywhere in the DOM: it has exactly 4 valid parent elements, of which 1 also has been obsoleted. Here are some examples of it in action:</p> <</span>aside</span>> </span> <</span>li</span>>first</</span>li</span>> </span> <</span>li</span>>second</</span>li</span>> </span></</span>aside</span>> </span> </span><</span>menu</span>> </span> <</span>li</span>>first</</span>li</span>> </span> <</span>li</span>>second</</span>li</span>> </span></</span>menu</span>> </span></code></pre> Quiz time: which of these is valid? Which of these isn't? None, both, either one? When you write the code, you don't know. It's only when you try and render the code in the browser that you'll receive feedback.</p> Okay fine, the answer to the last question was that <aside><li></code> is invalid, but <menu><li></code> is valid. Quiz number two (no peeking at the reference): what is rendered here?</p> <</span>ol</span>> </span> <</span>li </span>value</span>="</span>2</span>">second</</span>li</span>> </span> <</span>li </span>value</span>="</span>1</span>">first</</span>li</span>> </span></</span>ol</span>> </span> </span><</span>menu</span>> </span> <</span>li </span>value</span>="</span>2</span>">second</</span>li</span>> </span> <</span>li </span>value</span>="</span>1</span>">first</</span>li</span>> </span></</span>menu</span>> </span></code></pre> Okay, answer time: first one is valid, second one is not. But both will render, without error. It's just that in the first case value</code> is respected ("first, second") while in the second case it renders "second, first". Did you know that? I sure didn't.</p> As you're probably aware I'm not actually trying to teach you the rules of <li></code>, but just trying to show that this teeny tiny basic element has a host of interactions that will affect what you build. And we're only scratching the surface of this element. It also interacts with styling, event handlers, ARIA roles, and elements contained within. The fact that value</code> attributes on <li></code> are valid when nested within <ol></code>, but not within <menu></code> is just one interaction of thousands out there. Everyone gets things wrong all the time with this. And we need to do better. All these rules are specced, implemented, and (mostly) consistent. We need a way to automate this knowledge.</p> Type-Safe HTML</h2> I think we need to start by acknowledging that HTML is complex. Not in a deregatory way, HTML is also really useful, but as a statement of fact. There are more rules and interactions than anyone reasonable person can manage. Which means we shouldn't expect people to be the ones managing it.</p> Instead, I believe that HTML would benefit from type safety</em>. Right now the way HTML is validated is by shipping it to a browser, and then checking the console tab for errors. Or sometimes people run linters on it such as lighthouse</a> or pa11y</a>. Sometimes these things even happen in CI.</p> A shortcoming of this approach is that in order to validate whether you've done things right, you first have to load your web page into a browser engine, and only then you can validate whatever was rendered. But that's where it stops: it only validates what was rendered at that point. If your page has different components which can be in different states, or also has different pages, you need to ensure every one of those is validated. And that's often more work than it's worth, so people don't bother.</p> Instead the way I'm thinking about it is: what if we moved the validation to a compilation pass. That way we can prevent errors from occurring altogether. A compiler is able to reason about all states, and validate they're all valid, all in a blink of an eye. No browsers required. For people who're looking to reduce the amount of runtime issues in their websites: this is by far the most feasible way to actually get the number of HTML bugs down to zero.</p> This isn't the first time I'm writing about this: I first pitched this in 2019</a> because I didn't have the bandwidth to implement it myself. But Bodil did, who ran with it and subsequently published the typed-html</code> crate</a>. This is good; I'd love to see different approaches of this same idea spring up in Rust and in other languages. All that's needed to get started is an idea of what the resulting API should look like. And then write a compiler which takes .webidl</code> node definitions</a>, and outputs types in the right shape on the other end 2</a></sup>.</p> ^{2</sup> It'd be amazing if the W3C kept a repo with these definitions up to date themselves, so everyone else could just pull them in. But one can wish.</p> </div>}Type-Safe Semantic Represenation</h2> Once you have a base of checked HTML types that's pretty good. But that doesn't mean we're quite done yet. Remember when I said we weren't getting into ARIA roles? Well, we are now.</p> So see, if HTML is structure</em>, and CSS is representation</em>, then ARIA is "semantics". WAI-ARIA</a>, or "ARIA roles" for short, enables you to ascribe meaning to your structure. When you see a bunch of <li></code> items in text form, what are you looking at? Are they tabs? Are they a menu? Are they a dropdown? ARIA roles enable you to meaningfully distinguish between them. How many of these roles are there? Over 250, and they all have different properties and interactions.</p> Again: the path away from trouble here is by creating abstractions. Instead of having to remember which elements to apply the tablist</code></a>, tab</code></a>, and tabpanel</code></a> roles on, you should be able to create something akin to Swift's TabView</code></a> which encapsulates all of those differences for you.</p> This is a higher level of abstraction than just typed HTML provides. But it's equally important, since it's a requirement for useful keyboard navigation and accessibility. It'll take some more work than just plain typed HTML, and some more design work is required to incorparate best practices on how to actually structure elements since you can't just scrape a spec for definitions. But once you can reason about e.g. "tabs" or "feeds", building more complex things becomes a lot easier. Check out the ARIA Patterns page</a> to get a sense of the kind of stuff this could cover.</p> Embracing Compilation</h2> To recap so far: we've established that HTML and ARIA are complex</em> interfaces, that checking during compilation rather than at runtime has a lot of benefits, and that applying this to ARIA it would enable us to reason about elements at a higher level of abstraction.</p> I believe that in order to make this all make the most sense, the web is missing "sourcemaps for HTML". If we're going to be compiling a bunch of semantic components down to HTML, then devtools should enable us to reason about them as components</em>. When inspecting HTML in the DOM, we shouldn't have to look at a bunch of <div></code>, <p></code>, and <span></code> tags. But more meaningful things, such as <feed></code>, <profile></code>, and <calendar></code>.</p> The way I think this is that devtools are like having access to a debugger. When debugging a compiled program we usually don't reason in terms of MOV rax jax</code> and POWF bonk donk</code>. But the operations we actually wrote such as: pet_cat</code>, and big_stretch</code>. It's relatively infrequently that actual output is what we're interested in, but it's always there if you need it.</p> Some of you web heads might be wondering: "What about Web Components? - Isn't that the same?"</em> And, like, kind of! It's similar in how a reflection in a mirror is similar to the thing it reflects: similar at first sight, but actually everything is the opposite.</p> The point of the Custom Elements part of Web Components is that you can send <custom-tags></code> to the browser which are then interpreted at runtime to introduce interactivity, but preserve qualities such as cacheability of shared resources. A canonical example here is the relative-time-element</code></a> tag used by GitHub. What we're doing is the opposite: the HTML we send to the browser is already fully expanded, and doesn't need to interpret any JS to enable interactivity. What we want is for the browser to be able to interpret all of our <li></code> and <span></code> tags generated by our components as the <calendar></code> and <feed></code> elements that we've authored. This is much closer to source maps: associate the building blocks we generated with the semantic components they actually came from.</p> The similarity with custom elements comes back up when we take a look at how custom-elements are rendered in the browser's Elements tab:</p> <</span>relative-time </span> </span>tense</span>="</span>past</span>" </span> </span>datetime</span>="</span>2016-11-23T11:12:43-08:00</span>" </span> </span>data-view-component</span>="</span>true</span>" </span> </span>title</span>="</span>Nov 23, 2016, 8:12 PM GMT+1</span>"> </span> #shadow-root (open) </span> "6 years ago" </span> November 23, 2016 11:12 </span></</span>relative-time</span>> </span></code></pre> This contains all the information anyone needs to debug the element. And what I'd love is to be able to access this view for server-rendered components without ever needing to go use any JS. And the uses don't stop there either: frameworks such as React also reason in terms of components and require custom devtools</a> to re-surface that view. What we really need is a way to ascribe semantic meaning to our elements in a way that works for both server- and client-side applications.</p> Now, it could be that I've missed a capability that would enable doing exactly</em> this with custom elements in browsers today - or perhaps some other mechanism. It's been a minute since I last used custom elements. But if that's the case I'd love to learn about it, because in my opinion every framework under the sun should start making use of that. I know I will!</p> Conclusion</h2> In this post we've talked about the complexity of getting HTML right, how using a compiler could be used to mitigate those issues, what ARIA roles are, how we could model higher-level type-safe components, and finally what's missing in the browsers devtools to round this out.</p> I strongly believe that type-safety would be a boon for the browser space. People have already experienced some of it with the adoption of TypeScript, and I believe leaning further into types and type-safety would enable the web platform to become far more accessible - both for developers and users.</p> The only piece we can't directly control is the ability to teach browsers to reason about the semantic meaning of elements - almost like being able to declare source maps for the resulting HTML. In an alternate universe this is what Custom Elements</a> would have been. But I believe in order to make the web platform more accessible, we must make it possible to actually reason about <feed></code> and <calendar></code> as first-class items. And not just the <div></code> and <p></code> tags they're comprised of.</p> update (2023-04-07):</strong> As coincidence will have it, two weeks after publishing this post the chrome</a> and safari</a> teams began discussing the upcoming "declarative shadow DOM" feature. This seems like it will resolve the outstanding issues involving shipping semantically meaningful elements from the server.</p> update (2023-04-11):</strong> I published the html</code> crate</a> implementing most of this blog post. You can find the source on my GitHub at yoshuawuyts/html</a>.</p> Capability-Safety I: Prelude Tue, 24 Jan 2023 00:00:00 +0000 Introduction</h2> In 2021 Tmandry published the post: "Contexts and Capabilities"</a>. In it they present the idea of with</code>-clauses, providing a means of passing types such as allocators around in a more convenient way. The post goes into detail about the feature, but you can roughly think of it as a way to declare types which are live within a certain scope, and are automatically passed into functions which require them:</p> //! A usage-only example of `with`-clauses to create scoped allocators. </span>//! This example is somewhat incomplete, but hopefully gives enough of a sense </span>//! of what it would feel like to use `with`-clauses. </span> </span>// Before: using `#[feature(allocator_api)]` to allocate into a bump allocator. </span>let</span> bump = bumpalo::Bump::new(); </span>// Create a bump allocator </span>let</span> vec1 = Vec::new_in(&bump); </span>// Use `new_in` to allocate into the bump allocator </span>let</span> vec2 = Vec::new_in(&bump); </span> </span>// After: using `with`-clauses to allocate into a bump allocator. </span>with std::alloc::alloc = bumpalo::Bump::new() { </span>// Create a bump allocator for this scope </span> </span>let</span> vec1 = Vec::new(); </span>// `new` uses whichever allocator is currently active </span> </span>let</span> vec2 = Vec::new(); </span>} </span></code></pre> Capability-Safety</h2> While we were brainstorming about with</code>-clauses it got me wondering: Would it be possible to not only leverage with</code>-clauses as a means of just passing arguments around, but also as a means of guaranteeing encapsulation</em>?. Or differently put: if we can think of with</code>-clauses as a way of passing capabilities, would it be possible for us to make Rust "capability-safe" 1</a></sup>? I reached out to Sunfishcode</a> who has worked a lot on capabilities for WebAssembly, and for the past year or so we've been researching how we might be able to introduce a notion of capability-safety into Rust.</p> ^{1</sup> I'm intentionally not really going into detail about what "capability-safety" is exactly; a follow-up to this post is being written. But to not leave people entirely in the dark, let me try and summarize it: What if we had access to a kind of function which guarantees it will only ever operate over exactly</em> what it declared in its function signature and nothing else. This enables functions to not only describe what they can</em> do, but by omission now also what they can't</em> do. A lot of safety-critical systems would love to have access to that kind of expressiveness, but there are a bunch of other uses for it too. This intersects with a bunch of different aspects of the language, so expect us to go into minute detail about how we believe we can make this all work in future posts.</p> </div> The good news is: we think we have a plausible design - one which not only enable capability-safety, but crucially would so without breaking any of Rust's stability guarantees. We've consulted with members of both lang and libs, and at least from a technical point of view it seems like it could work. But if we ever want to turn it into a formal Rust language proposal, we not only need the design to be plausible</em>, we also need to show that it's something which is beneficial</em>. Something which solves real issues people experience, and something which people would be excited to adopt.</p>}The plan</h2> You can consider this post a statement of intent. Sunfish and I are committed to figuring out how we can actually talk about capability-safety in a way that makes sense. I've tried and failed to write something coherent post about this a couple of times now. But we believe it's time to actually start publishing about this. So that's exactly what we'll be doing!</p> We'll start explaining by what capabilities are, why they're useful, what capability-safety is, which guarantees it provides, introduce you to our design, and then work through the various interactions. There's not a lot of material on capability-safety available, so the real challenge for us is to figure out the right framing and examples to have it actually make sense.</p> But we're convinced we can actually make it happen. Sunfish has already started writing about some these topics which you can check out here:</p> No ghosts!</a> (2022-03-15)</li> What is a capability?</a> (2022-11-21)</li> </ul> His next post will be about capability-safety. Stay tuned for that!</p> Thanks to Sunfishcode</a> for help reviewing this post!</em></p> domain-specific error macros Tue, 17 Jan 2023 00:00:00 +0000 When I write custom errors in a project, I also like to write a few small error macros to accompany them. In my opinion this can make error handling just a little nicer to use. In this post I briefly want to talk about domain-specific error macros such as ensure!</code>, what they're useful for, and the io-ensure</code></a> prototype crate I've written which I'm propose for inclusion in the stdlib next time I get the chance.</p> What is ensure?</h2> If you use Rust you're probably familiar with assert!</code></a>: it checks a condition, and if the condition doesn't match it panics. This often looks something like this:</p> let</span> condition = </span>1 </span>== </span>1</span>; </span>assert!(condition, "</span>numbers should equal themselves</span>"); </span></code></pre> This works well if you're writing tests, or validating invariants - but when you're validating input at runtime, instead of panicking you usually probably want to return errors instead. And the error equivalent to assert!</code> is ensure!</code>:</p> fn </span>validate_buffet</span>(</span>s</span>: &</span>str</span>) -> io::Result<()> { </span> ensure!(s.</span>starts_with</span>("</span>tuna</span>"), "</span>the first course on the cat menu should be tuna</span>"); </span> Ok(()) </span>} </span></code></pre> When discussing "ensure-style macros", these are typically the ones included:</p> format_err!</code>: construct a new instance of an error using a format string</li> bail!</code>: unconditionally create and return an error</li> ensure!</code>: create and return an error if the condition doesn't match</li> ensure_eq!</code>: create and return an error if two expressions don't match</li> ensure_ne!</code>: create and return an error if two expressions match</li> </ul> bail</code> is kind of like panic</code>, and ensure</code> is kind of like assert</code>. You can't really construct and pass around panics, so there is no panic-equivalent to format_err</code>. Also for simplicity I'm omitting matches</code> variants, and errors probably shouldn't be debug-only, so no debug</code> variants either.</p> Pairing errors with macros</h2> I believe it's good practice that if you're defining your own errors, to also define your own error handling macros. Especially if it's intended to be extended by third parties, it can make it significantly nicer to work with.</p> The key here is that you can use ensure!</code> macros to encode domain-specific logic with. Say you're writing an HTTP framework, you probably want all of your errors to include an HTTP status code. While if we're dealing with IO, we probably want to map back the underlying OS reason:</p> // Example using `http_types::ensure!` </span>ensure!(user.</span>may_access</span>(resource), StatusCode::</span>403</span>, "</span>user is not authorized to access {resource}</span>"); </span> </span>// Example using `io-ensure` </span>ensure_eq!(a, b, ErrorKind::Interrupted, "</span>we are testing the values {} and {} are equal</span>", a, b); </span></code></pre> new keywords</h2> Work is happening in Rust on creating new language features to reason about errors. areweyeetyet</a> provides an overview of this, but the tldr is that we want to have a way to define "fallible functions" from which you can "throw" errors. Also: no more Ok(())</code> at the end of every function.</p> Having access to a throw</code> (or yeet</code>) keyword means we'll have a canonical way of returning an error from a function. ?</code> to re-throw errors, throw</code> to throw new ones 1</a></sup>. But the bail</code> macro does something very similar already, and has the downside of having to be defined for every error type:</p> ^{1</sup> I'm using "throw" in the Swift sense of the keyword: "checked exceptions". Not the JS/Java/etc "unchecked exceptions" throwing; I think we all like our Rust errors to be typed.</p> </div>}// current </span>return </span>Err(io::Error::new(e.</span>kind</span>(), format!("</span>{path}</span>: </span>{e}</span>")).</span>into</span>()); </span> </span>// using `format_err!` </span>return </span>Err(io::format_err!(e.</span>kind</span>(), "</span>{path}:{e}</span>").</span>into</span>()); </span> </span>// using `bail!` </span>io::bail!(e.</span>kind</span>(), "</span>{path}:{e}</span>")); </span> </span>// if we had `throw` in the language (feature composition!) </span>throw io::format_err!(e.</span>kind</span>(), "</span>{path}:{e}</span>"); </span></code></pre> What I'm thinking here is that it makes sense to include bail</code> if you're writing your own libraries and applications. But probably less for the stdlib, since whatever goes in there will need to remain stable forever, and having bail</code> macros + throw</code> keywords is probably a bit too much.</p> Stdlib</h2> So I've written a quick prototype showing: "What if std::io</code> had domain-specific macros" and published it to crates.io: io-ensure</code></a>. Is it great? I'm not sure. I think it's a decent quality of life improvement, and perhaps it might make sense to include. Just like assert</code> can sometimes make it easier to work with panics, ensure</code> might sometimes make it easier to work with errors. Here's some random examples of IO error uses I translated from the rust-lang/rust</code> repo:</p> This example omits an extra inline call to format</code>, removing some of the nesting:</p> // before </span>fs::create_dir_all(parent).</span>map_err</span>(\|</span>e</span>\| { </span> io::Error::new( </span> e.</span>kind</span>(), </span> format!("</span>IO error creating MIR dump directory: </span>{parent:?}</span>; </span>{e}</span>"), </span> ) </span>})?; </span> </span>// with io macros </span>fs::create_dir_all(parent).</span>map_err</span>(\|</span>e</span>\| { </span> io::format_err!(e.</span>kind</span>(), "</span>IO error creating MIR dump directory: {parent:?}; {e}</span>"); </span>})?; </span></code></pre> rust/compiler/rustc_middle/src/mir/pretty.rs</em></a> ←</p> If librustdoc</code> also had its own error macros, we could imagine we could compose it with IO error macros like so, saving us from some nesting and branching:</p> // current </span>if</span> nb_errors > </span>0 </span>{ </span> Err(Error::new(io::Error::new(io::ErrorKind::Other, "</span>I/O error</span>"), "")) </span>} </span>else </span>{ </span> Ok(()) </span>} </span> </span>// with `librustdoc::ensure!` + `io::format_err!` </span>ensure!(nb_errors > </span>0</span>, io::format_err!(io::ErrorKind::Other, "</span>I/O error</span>")); </span>Ok(()) </span></code></pre> rust/src/librustdoc/html/render/context.rs</em></a> ←</p> This call to format</code> with the referencing and subslicing looks rather spicy. I'm not super sure what's going on there, but presumably we should be able to avoid it already? Either way, that's still an extra inline format</code> call gone:</p> // current </span>Err(io::Error::new( </span> io::ErrorKind::Uncategorized, </span> &format!("</span>failed to lookup address information: </span>{detail}</span>")[..], </span>)) </span> </span>// with `io::ensure!` </span>Err(io::format_err!( </span> io::ErrorKind::Uncategorized, </span> "</span>failed to lookup address information: {detail}</span>" </span>)) </span></code></pre> rust/library/std/src/sys/unix/net.rs</em></a> ←</p> Conclusion</h2> In this post we've talked about domain-specific error macros, what they are, what they're used for, and shown some examples of how they could be used for io::Error</code> construction in the stdlib.</p> Overall it's just a little convenience pattern which makes working with errors nicer. Just like assert!</code> is a small quality of life improvement which makes panics nicer to use; ensure!</code> is a small quality of life improvement which can make errors nicer to use.</p> Rust should own its debugger experience Thu, 12 Jan 2023 00:00:00 +0000 Introduction</h2> 40% of Rust Developers</a> believe Rust's debugging experience could use improvement. And that's not surprising: when we write code we make assumptions, and sometimes we assume wrong. This leads to bugs, which we then track down and fix. This is what we call "debugging", and is a core part of programming. The purpose-built tools which help us with debugging are called "debuggers".</p> Unlike the Rust compiler, the Rust project doesn't actually provide a "Rust debugger". Users of Rust are instead expected to use a third-party debugger such as gdb</code>, lldb</code>, or windbg</code> to debug their programs. And support for Rust in these debuggers is not always great</em>. Basic concepts such as "traits", "closures", and "enums" may have limited support. And debugging async code, or arbitrary user-defined data structures may be really hard if not impossible. This limits the utility of debuggers, and in turn limits the Rust user's debugging experience.</p> Integration done right</h2> When we interact with the "rust compiler", it's often through either cargo build</code>, or Rust-Analyzer in VSCode 1</a></sup>. The fact that a large chunk of the compiler is in fact LLVM is for all intents and purposes just an implementation detail. Even Rust-Analyzer powering the VS Code features is not something you need to think about most of the time.</p> This is how it should be. When you install Rust, the right version of LLVM gets bundled in. The Rust project is responsible for LLVM doing the right things for Rust, and we'll sometimes even float patches on a fork to fix things the project depends on. Users should never become aware of these details. We're even at a point where we might replace LLVM with a different backend in certain scenarios 2</a></sup>, and the only thing users should notice are faster build times.</p> ^1</sup>VS Code has 75% market share apparently. (src)</a></p> </div> ^{2</sup> I'm talking about Cranelift for debug builds here.</p> </div>}Debuggers today</h2> Debuggers have a very different story today. The Rust project doesn't really provide a "debug workflow", meaning Rust users are made largely responsible for figuring that out themselves. This means they need to find a debugger, install it, and figure out how to run it. Whatever Rust support the debugger has is what they get. And hopefully Rust is enough on debugger author's radars that support for it improves over time. It's another question when newer versions of a debugger may become available to Rust users too. As a project we can contribute some patches, but at a fundamental level we don't have any control</em> over these tools. Which means we don't have any control over the resulting user experience.</p> Work is happening though to enable the output of the Rust compiler to be better understood by third-party debuggers. For example RFC 3191: Debugger Visualizer</a> has been accepted which enables Rust users to author debugger-specific visualizer scripts. This enables library authors to define debugger-specific scripts which can provide more details about types. This is a very pragmatic fix which will enable, say, the stdlib to immediately start providing a better UX for existing debuggers. But we can't reasonably expect all Rust projects to define debugger visualizations for all of their data structures, so as a general solution it has some pretty clear limitations.</p> Towards a coherent experience</h2> In my opinion the Rust project needs to rethink its relationship to the debugger user experience. Rather than something that we leave up to users to figure out, it should be something that the Rust project is in charge of providing to Rust users. If we want the Rust debugging experience to improve, we in the project need to be the ones responsible for providing that experience.</p> I'm not suggesting that we start writing debuggers from scratch. Instead I'm suggesting we start packaging and distributing existing debuggers for all platforms, together with plugins which extend those debuggers with Rust-specific functionality. This will enable us to teach those debuggers about things like traits, enums, and even async 3</a></sup>. And we can ensure that the debuggers + plugins are updated regularly, and work correctly. Just like with LLVM, the specific debugger should become just an implementation detail.</p> ^{3</sup> We can work on ways to extend this to work with ecosystem runtimes as well.</p> </div>}Which debuggers to package 4</a></sup>, and how to distribute them are details I'm confident we can figure out. We've done this before for other tools, so that shouldn't be too different. The main question is how to expose the debugger, and for that I would propose we start with the Debug Adapter Protocol (DAP)</a>, LSP's less-famous sibling. Just like we use LSP to provide IDE features in an IDE-agnostic way, we can use DAP to provide debugger support in an IDE-agnostic way.</p> ^{4</sup> Maybe we don't even want to package certain platform-specific debuggers, and instead we just want to distribute bindings. Maybe different platforms should make different decisions. Or perhaps people should be able to choose. These are the kinds of details I expect experts to want to weigh in on.</p> </div>}I believe by exposing the debugger as a server, we can enable others to write tooling for it. For example: we could write standalone VSCode extensions which integrate with it, or perhaps contribute a DAP integration directly to Rust-Analyzer 5</a></sup>. Perhaps some people prefer an editor-based debugger flow; which should be possible to build against DAP. And should DAP provide limitations, we can always write extensions with the intent to later upstream them - which is again something we do with LSP as well.</p> ^{5</sup> I defer to the Rust-Analyzer team to comment on what the best course of action here might be.</p> </div>}A really interesting idea in the Rust debugger space has been the approach of plugging a MIR interpreter into a debugger as an extension, which is then able to evaluate Debug</code> impls at runtime. This has been prototyped</a> and shown to be possible. In my opinion that's the ideal solution because it doesn't require any user code to be changed, and it's also exactly the kind of thing that the Rust project can do which external debugger projects can't. Just write your Debug</code> impls, and the debugger will figure out how to interpret them. Making this practical will not be without challenges, but those are mostly technical - and technical talent is something we're not short on in the Rust project.</p> Conclusion</h2> Those are my thoughts on how I think we can structurally improve the debugging user experience in Rust. I believe that by making the Rust project responsible for the debugger experience we can provide a fundamentally better experience than any external debuggers ever could. And I would be excited if this was something the Debugging WG</a> would seek to pursue.</p> One last note before publishing: Debugger extensions in VS Code also have support for profiling and even REPL-like evaluation</a>. I think this is part of DAP too, or some extension to it at least. I feel very similar about profiling like I do about debugging: the Rust project should be responsible for providing a premier user experience. But we have to start somewhere, and I think if we can build the integrations needed to support debugging, we can eventually grow the scope to become responsible for profiling too. Go</a> and JS</a> have this already, and I'd love for Rust to catch up.</em></p> Thanks to Michael Woerister</a> for reviewing!</em></p> State Machines III: Type States Mon, 02 Jan 2023 00:00:00 +0000 I've been writing a bit about state machines recently. In my last post I explained how we could potentially leverage arbitrary self-types and anonymous enums to support type-level state machines as a first-class construct. Which is cool, but it had some readers asking: "How does this improve on the existing type state pattern?"</em> Which is fair!</p> Well, to start off by answering that question: it's only after authoring that post that I learned that "type states" have a name! I was familiar with the pattern, but I didn't realize it had a name. If you'd like to learn more about them, Will Chrichton gave an excellent presentation about API design which also covers type states</a>.</p> Though type states are powerful, they're not the easiest to use - and can be limited in expressivity at times. In this post I want to show what some of those limitations are, and how we can over come them by encoding them using first-class type-level state machines instead.</p> editor's note: I tried capping this post off at 500 words and failed. I'm sorry, here's 2500 words instead.</em></p> Setting the stage</h2> We'll continue using the example we've been using in the past two posts; a traffic light which can transition between states. We'll take the simpler variant of that for this post:</p> green can transition to yellow</li> yellow can transition to red</li> red can transition to green</li> </ul> If we want to encode it with the type-state pattern, we have to create separate structs for each of the states, and then parameterize the traffic light struct over the state. This is roughly what that looks like (playground</a>):</p> use </span>std::marker::PhantomData; </span> </span>struct </span>Green; </span>struct </span>Yellow; </span>struct </span>Red; </span> </span>struct </span>TrafficLight<S> { </span> </span>_phantom</span>: PhantomData<S> </span>} </span> </span>impl </span>TrafficLight<Green> { </span> </span>fn </span>new</span>() -> TrafficLight<Green> { </span> TrafficLight { _phantom: PhantomData } </span> } </span>} </span> </span>impl </span>TrafficLight<Green> { </span> </span>fn </span>next</span>(</span>self</span>) -> TrafficLight<Yellow> { </span> TrafficLight { _phantom: PhantomData } </span> } </span>} </span> </span> </span>impl </span>TrafficLight<Yellow> { </span> </span>fn </span>next</span>(</span>self</span>) -> TrafficLight<Red> { </span> TrafficLight { _phantom: PhantomData } </span> } </span>} </span> </span>impl </span>TrafficLight<Red> { </span> </span>fn </span>next</span>(</span>self</span>) -> TrafficLight<Green> { </span> TrafficLight { _phantom: PhantomData } </span> } </span>} </span> </span>fn </span>main</span>() { </span> </span>let</span> light = TrafficLight::new(); </span>// TrafficLight<Green> </span> </span>let</span> light = light.</span>next</span>(); </span>// TrafficLight<Yellow> </span> </span>let</span> light = light.</span>next</span>(); </span>// TrafficLight<Red> </span> </span>let</span> light = light.</span>next</span>(); </span>// TrafficLight<Green> </span>} </span></code></pre> This makes it so that if we attempt to call to_red</code> when we have a TrafficLight<Green></code>, the compiler will disallow it. Type states allow us to encode program invariants into the type system, preventing it from causing runtime bugs.</p> Where type states run into trouble</h2> Type states become tricky when you start wanting to combine them with runtime logic. After all, a traffic light example is nice: but what if we're actually trying to run it? Take for example the following code, which can't be neatly represented using type states:</p> use </span>futures_lite::prelude::; </span>use </span>futures_time::prelude::; </span>use </span>futures_time::time::Duration; </span>use </span>futures_time::stream; </span> </span>let mut</span> light = TrafficLight::new(); </span>let mut</span> interval = stream::interval(Duration::from_secs(</span>20</span>)) </span>while let </span>Some(_) = interval.</span>next</span>().await { </span> light = light.</span>next</span>(); </span>// Cycle to the next state every 20ms </span>} </span></code></pre> We want to switch to the next state</code> every 20s, but this won't compile because TrafficLight<Green></code> and TrafficLight<Yellow></code> aren't the same type, so they can't be re-assigned to the same value. In order to do so we can match the type states back to an actual enum, and wrap the type-state variant in it. This could look something like this (playground</a>):</p> use </span>futures_lite::prelude::; </span>use </span>futures_time::prelude::; </span>use </span>futures_time::time::Duration; </span>use </span>futures_time::stream; </span> </span>enum </span>TrafficLightWrapper { </span> Green(TrafficLight<Green>), </span> Yellow(TrafficLight<Yellow>), </span> Red(TrafficLight<Red>), </span>} </span> </span>impl </span>TrafficLightWrapper { </span> </span>pub fn </span>new</span>() -> </span>Self </span>{ </span> TrafficLightWrapper::Green(TrafficLight::new()) </span> } </span> </span> </span>pub fn </span>next</span>(</span>self</span>) -> </span>Self </span>{ </span> </span>match </span>self </span>{ </span> TrafficLightWrapper::Green(inner) => TrafficLightWrapper::Yellow(inner.</span>next</span>()), </span> TrafficLightWrapper::Yellow(inner) => TrafficLightWrapper::Red(inner.</span>next</span>()), </span> TrafficLightWrapper::Red(inner) => TrafficLightWrapper::Green(inner.</span>next</span>()), </span> } </span> } </span>} </span> </span>let mut</span> light = TrafficLightWrapper::new(); </span>let mut</span> interval = stream::interval(Duration::from_secs(</span>20</span>)) </span>while let </span>Some(_) = interval.</span>next</span>().await { </span> light = light.</span>next</span>(); </span>} </span></code></pre> This is a non-trivial amount of boilerplate, just to work with code which uses the type state pattern at runtime. 50 lines to encode just 3 states is probably too much to be practical for most uses, even if the resulting API is flexible enough to be used in a variety of scenarios. We should be able to do better than this!</p> Doing better with enums</h2> Now, say we had access to enum-variant-types</code></a> (enum variants as first-class types), and arbitrary-self-types</code></a>, we could imagine a reformulation of the traffic light as follows:</p> //! A basic traffic light state machine </span>//! Novel language features: enum-variant-types, arbitrary-self-types </span> </span>enum </span>TrafficLight { </span> Green, </span> Yellow, </span> Red, </span>} </span> </span>impl </span>TrafficLight { </span> </span>pub fn </span>new</span>() -> </span>Self::</span>Green { </span> </span>Self</span>::Green </span> } </span> </span> </span>pub fn </span>next</span>(</span>self</span>: </span>Self::</span>Green) -> </span>Self::</span>Yellow { </span> </span>Self</span>::Yellow </span> } </span> </span> </span>pub fn </span>next</span>(</span>self</span>: </span>Self::</span>Yellow) -> </span>Self::</span>Red { </span> </span>Self</span>::Red </span> } </span> </span> </span>pub fn </span>next</span>(</span>self</span>: </span>Self::</span>Red) -> </span>Self::</span>Green { </span> </span>Self</span>::Green </span> } </span>} </span> </span>fn </span>main</span>() { </span> </span>let mut</span> light = TrafficLight::new(); </span>// TrafficLight<Green> </span> light = light.</span>next</span>(); </span>// TrafficLight<Yellow> </span> light = light.</span>next</span>(); </span>// TrafficLight<Red> </span> light = light.</span>next</span>(); </span>// TrafficLight<Green> </span>} </span></code></pre> Because TrafficLight</code> is now a single type, it can be assigned to the same variable without a problem. Paving over some details, we could imagine the following could then work as well:</p> // ..imports.. </span> </span>let mut</span> light = TrafficLight::new(); </span>let mut</span> interval = stream::interval(Duration::from_secs(</span>20</span>)) </span>while let </span>Some(_) = interval.</span>next</span>().await { </span> light = light.</span>next</span>(); </span>// Cycle to the next state every 20ms </span>} </span></code></pre> And that, in a nutshell, is why I believe that access to first-class state machines at the type level would be helpful. It would enable us to use type states in more places, which in turn would allow us to create more robust abstractions, which yet in turn lead to fewer bugs.</p> Embedding in structs</h2> Being able to treat enums as type-level state machines has other benefits as well: it makes it so when you embed a state machine in another type, the state itself doesn't necessarily need to be observable from the outside either. Take the following code:</p> struct </span>Green; </span>struct </span>Yellow; </span>struct </span>Red; </span>struct </span>TrafficLight<S> { </span> </span>_phantom</span>: PhantomData<S> </span>} </span> </span>// A new struct containing multiple traffic lights </span>struct </span>Crosswalk<S1, S2> { </span> </span>light1</span>: TrafficLight<S1>, </span> </span>light2</span>: TrafficLight<S2>, </span>} </span> </span>impl</span><S1, S2> Crosswalk<S1, S2> { </span> </span>pub</span> async </span>fn </span>run</span>(</span>self</span>) { </span> </span>// loop here. </span> } </span>} </span></code></pre> The internal states for each of the traffic lights necessarily becomes externally observable, even if no external methods ever need to observe it. The internal state leaks</em> to the external API surface, making it a leaky abstraction. We could of course define the wrapper type we saw earlier again. But say we did have first-class state machines, we could encode it directly like this instead:</p> enum </span>TrafficLight { </span> Green, </span> Yellow, </span> Red, </span>} </span> </span>// A new struct containing multiple traffic lights </span>struct </span>Crosswalk { </span> </span>light1</span>: TrafficLight, </span> </span>light2</span>: TrafficLight, </span>} </span> </span>impl </span>CrossWalk { </span> </span>pub</span> async </span>fn </span>run</span>(</span>self</span>) { </span> </span>// loop here. </span> } </span>} </span></code></pre> We could still choose to surface the enums to the outside world if we wanted to (more on that later in this post). But there is a real benefit to not having to if you don't want to, without needing to define pages of boiletplate!</p> Enum method unification</h2> One assumption I've made is that if you have an enum with three states, and you define a method for all three states, then that method should become available even if we don't know which state we're in.</p> Using match</code> we can always find out what state we're in by inspecting the enum's tag, meaning we can always figure out what state we're in and what state we should transition to, at runtime.</p> This also means that, say, if a method is only available for 2/3 of the states, as long as we can match</code> down to those states, the method should be available:</p> enum </span>RGB { Red, Green, Blue } </span> </span>impl </span>RGB { </span> </span>fn </span>test</span>(</span>self</span>: </span>Self::</span>Green \| </span>Self::</span>Blue) { </span> println!("</span>can print!</span>"); </span> } </span>} </span> </span>fn </span>try_to_print</span>(</span>rgb</span>: RGB) { </span> </span>match</span> rgb { </span> </span>RGB</span>::Green \| </span>RGB</span>::Blue => rgb.</span>print</span>(), </span>// `print` is defined for both states here </span> _ => { </span>/* do nothing / </span>} </span> } </span>} </span></code></pre> Enums as type params</h2> Even if we could hide the internal state sometimes, doesn't mean we'd always want to do it. I think one other benefit of "enums as state machines" in this model would be just to improve the existing type-state pattern. Take for example my tasky</code></a> crate. In v3.0.0 we distinguished between a "local builder" and "non-local builder" at the type level. This was needed so we could add different IntoFuture</code> implementations to them. This is how they're defined:</p> /// Sealed trait to determine what type of builder we got. </span>mod </span>sealed { </span> </span>pub trait </span>Kind {} </span>} </span> </span>/// A local builder. </span>pub struct </span>Local; </span>impl </span>sealed::Kind </span>for </span>Local {} </span> </span>/// A nonlocal builder. </span>pub struct </span>NonLocal; </span>impl </span>sealed::Kind </span>for </span>NonLocal {} </span> </span>/// Task builder that configures the settings of a new task. </span>pub struct </span>Builder<Fut: Future, K: sealed::Kind> { </span> </span>kind</span>: PhantomData<K>, </span> </span>future</span>: Fut, </span> </span>builder</span>: ..., </span>} </span></code></pre> That's a lot of work to encode the state 1</a></sup>. And just from glancing over the rustdoc output you'd never know that that's what this is for. Instead it'd be far nicer if we could define typestates using enums, and not actually have to bother with sealed traits and disjointed structs:</p> ^{1</sup> A note on sealed traits: this is the first time in the post we're using them, but they're useful to ensure that only types defined within the crate can ever be passed to it. If we had support for concrete types in bounds + enums as first-class types, then we'd probably have less need for sealed traits to be used in this way.</p> </div>}enum </span>BuilderKind { </span> Local, </span> NonLocal, </span>} </span> </span>/// Task builder that configures the settings of a new task. </span>#[</span>derive</span>(Debug)] </span>pub struct </span>Builder<Fut: Future, K: BuilderKind> { </span> </span>kind</span>: K, </span> </span>future</span>: Fut, </span> </span>builder</span>: ..., </span>} </span></code></pre> I believe doing this would require us to enable using concrete types as type-bounds. I believe this is something which comes up repeatedly, and we do want to resolve eventually for other reasons as well. Perhaps "better typestates" could be added to that list as well now?</p> Enum variant tracking</h2> Another feature which would probably be helpful here would be something I call "enum variant tracking". Take for example the following code, which inserts a value into Option</code> if it doesn't exist, and then provides us with mutable access to it:</p> let mut</span> option = None; </span>if</span> option.</span>is_none</span>() { </span> option = Some(</span>12</span>); </span>} </span>let</span> num: &</span>mut u32 </span>= option.</span>as_mut</span>().</span>unwrap</span>(); </span>// may panic </span></code></pre> Sure, Option::unwrap_or_else</code> exists and does the same. But assume for a second we've defined our own Option</code>, and we need to implement our own. We should never</em> have to unwrap</code> here, since we know from the code that we're guaranteed</em> to always have a Some</code> variant here. And the is_none</code> check is redundant here, since we're guaranteed to always have a None</code> variant to start with.</p> The idea is that if we know</em> for a fact that we're dealing with a specific variant, we should be able to skip the ceremony and checks to access that specific variant. Instead of just knowing we have an Option</code>, with "enum variant types" we should be able to accurately track that we in fact start with an Option::None</code>, which always becomes Option::Some</code> after. The latter might be a bit more tricky, but given both option</code> and is_some</code> are const</code>, we could imagine that the compiler should be able to always know this.</p> To clarify though: while I think this feature might make sense from a theoretical perspective, it may not make sense from a practical one</em>. I'm not trying to argue we should or shouldn't support this, more that it might be within the realm of possibility if we get "enum variant types". This feature would also only be intended for local reasoning only. Nothing should ever cross function boundaries; other features should be available to enable us to more granularly annotate things. I think if we ever start to get serious about wanting to allow people to write panic-free code, this may become interesting. I'm kind of reminded of the (unstable) Error::into_ok</code></a> method, which is gated behind the unwrap_infallible</code> feature on nightly.</p> let mut</span> option = None; </span>if</span> option.</span>is_none</span>() { </span> option = Some(</span>12</span>); </span>} </span>let</span> num: &</span>mut u32 </span>= option.</span>as_mut</span>().</span>into</span>(); </span>// will never panic </span></code></pre> narrowing of state modifications</h2> Something we haven't yet covered, but probably should is the fact that all the state transition things we've talked about only work with owned</em> types. They don't at all work with mutable references. When we're calling next</code> on an iterator we get a &mut Self</code> type. When transitioning between state machines, wouldn't it make sense for the state transitions to be able to mutate Self</code> rather than always having to return a value?</p> I've saved this one for last, because this is where I really start to hesitate to be honest. All the features we've covered so far are in my opinion fairly natural extensions to what we already have. Subsets of enums, being able to use enums in more places, being able to narrow our views. What we're talking about here feels much further out there</em>, and so I'm also way less sure whether it's a good idea.</p> The basic premise is as follows, we want to enable the following to work:</p> fn </span>main</span>() { </span> </span>let mut</span> light = TrafficLight::new(); </span>// TrafficLight<Green> </span> light.</span>next</span>(); </span>// TrafficLight<Yellow> </span> light.</span>next</span>(); </span>// TrafficLight<Red> </span> light.</span>next</span>(); </span>// TrafficLight<Green> </span>} </span></code></pre> This means we'd need to find a syntax in the view type to not only describe what types go in, but also what the type will transition to. Niko's view type post didn't cover this, and my previous state machine post didn't cover this either. But at least conceptually, it seems to me that if we would enable a narrowing of function parameters and function return types, we might also want to provide a way to express a narrowing of value modifications.</p> I'm completely just riffing here, but maybe something like this? This is supposed to indicate that after the function returns, the value of self</code> will be Self::Yellow</code>.</p> impl </span>TrafficLight { </span> </span>pub fn </span>next</span>(</span>self</span>: </span>Self::</span>Green -> </span>Self::</span>Yellow) { </span> </span>Self</span>::Yellow </span> } </span>} </span></code></pre> I'm really unsure about this. But I think if we end up doing anonymous enums and view types, then this might be a question we may want to find an answer for.</p> Language feature overview</h2> To recap: here is the list of language features we've covered in this series so far. None of them are quite enough to bring us all the way, but if we combine them they might unlock the ability to improve the status quo of type states:</p> enum variant types</strong>: Enable reasoning about enum members as first-class types. We can't specify say that a method returns a specific enum member. We can only ever return the full enum.</li> arbitrary self-types / enum view-types</strong>: Enable declaring Self: SomeEnum::Member</code>, allowing us to declare methods which are only available for certain enum variants.</li> anonymous enums / enum subsets</strong>: To enable us to declare sub-sets of enums using syntax like: Member1 \| Member2</code>. For example, we may want to declare that a certain method can only return a specific sub-set of errors, or specific network status codes.</li> enum member unification</strong>: Say a method next</code> is declared for SomeEnum::Blue</code> and SomeEnum::Red</code>. If we obtain Blue \| Red</code> then we should be able to call that method in either case. From the perspective of the language it should be no different than if we'd have an enum with just those two members, and we'd have the method implemented for that entire enum.</li> enum variant tracking</strong>: If we know</em> for a fact that we're dealing with a specific variant, we should be able to skip the ceremony and checks to access that specific variant.</li> narrowing of state modifications</strong>: If we end up with anonymous enums and view types, we'll have syntax to express narrowings of both owned input and output function params. If that's the case, we may also find ourselves wanting to express narrowings of in-out params, which requires a way to express.</li> concrete type bounds</strong>: So we can use where S: SomeEnum::Member</code> directly as a bound, where State</code> is an enum. Currently using sealed traits + individual structs seems like the best way to approximate this.</li> </ul> Conclusion</h2> We've covered quite a number of language features so far. Just to be clear: I'm not trying to say that we should be implementing any of this, and even if we decided to: that we should be prioritizing this. I'm more interested in thinking out loud about how we could improve on type states, and summarizing which language features could be implemented to bring us closer to that.</p> With that in mind, I hope this made for an interesting read. Thanks for reading, and happy new year!</p> Thanks to Vagrant</a> for proof reading this post!</em></p> 2022 in review Fri, 23 Dec 2022 00:00:00 +0000 It's pretty quiet at work right now, and every other application on my phone is telling me it's time to look back. So why not lean in and look back at the last 12 months. Not some big reflection time, but to share some things I've done and liked. I've published about a novel's worth of work in the past year, and probably read a multitude of that. So if you're looking to read some tech related things over the holidays, maybe you'll find something fun in this list!</p> My own work</h2> Here are things I've worked on and written about this year:</p> Rust Language Work</h3> Announcing the Keyword Generics Initiative</a>: Where we announce the creation of the "keyword generics initiative", a new group under the language team looking to add a new type of generic to the language 1</a></sup>.</li> More Enum Types</a>: We discuss what potentially the enum-equivalent of tuples and impl trait</code> could look like. This could be useful, for say, easier application error handling.</li> State Machines II</a>: We take a look at typestates2</a></sup> in Rust, and explore a way to make them a first-class citizen in Rust by leveraging arbitrary-self-types, enums as first-class types, and anonymous enums. The biggest innovation here is that we'd have a way to prevent them from leaking to enclosing types by using enums instead of generics.</li> Inline Crates</a>: We discuss the difference between "crates" and "modules", and explore what feature parity between the two could look like, and what challenges enabling that would bring.</li> Keywords I: Unsafe Syntax</a>: Reflecting on the various meanings of the unsafe</code> keyword in Rust.</li> Keywords II: Const Syntax</a>: Reflecting on the various meanings of the const</code> keyword in Rust.</li> </ul> ^{2</sup> I didn't know "type states" had a name until after I wrote the post, so they're not mentioned by name in the post.</p> </div>}^{1</sup> I'm planning to write an update on the work we've done so far after the holidays. Stay tuned for that I guess?</p> </div>}Async Work</h3> Why Async Rust</a>: In which I explain what async Rust is, which features it provides, and how we can meaningfully talk about whether it makes sense to adopt.</li> Futures Concurrency III: Select!</a>: A deep-dive into the async select!</code> macro, discussing its various shortcoming, and looking at an alternative structured concurrency operator which doesn't share these shortcomings.</li> Safe pin projections through view types</a>: Goes into some detail on how we could leverage the "view types" proposal to make pin projections safe. Not quite a full proposal, but intended to give a first impression of what this might look like.</li> Futures Concurrency IV: Join Ergonomics</a>: We take a look at Swift's async let</code> feature, and how we can enable similar semantics in Rust's libraries using IntoFuture</code>.</li> Enhancing .await</code> with IntoFuture</code></a>: The release 1.64 release notes cover the stabilization of IntoFuture</code>, and show some cool things you can do with it.</li> Async-Iterator crate</a>: A short PSA that I tried the unstable "async fn in traits" feature, and published a crate which makes use of it.</li> Postfix Spawn</a>: A first look at the nature of async Rust's spawn</code>-family of APIs, and how we can make them behave in a structured manner. I wrote this in a fever-dream, and probably should write a follow-up.</li> Async Cancellation II: Time and Signals</a>: A first look at a newly created suite of time-based APIs, combining concurrency and cancellation to create a cohesive set of async control flow constructs.</li> </ul> Miscellaneous</h3> Rust 2023</a>: My thoughts and vision on the Rust project, late 2022 edition.</li> </ul> Other's work</h2> Here are things I've read this year which were a highlight for me. Some of it was published this year, other things are from earlier years. But either way, these were highlights for me:</p> Rust</h3> "Memory Safe Languages in Android 13" by Vander Stoep (Google)</a>: provides compelling evidence that adopting Rust does in fact meaningfully reduce the number of memory safety bugs in codebases. And crucially: significantly reduces the number of high/critical vulnerabilities. Imo this is one of the most important reports written about Rust in recent years.</li> "Rust Atomics and Locks" by Mara Bos</a>: While technically not yet published, I did get a chance to read the pre-release version, and go in-depth on some of the topics in the book with Mara over the past year. I haven't yet finished the book, but I already feel like I understand atomics so much better now (read: at all), and in my opinion this is a must-have for everyone looking to go going beyond the basics in Rust.</li> "Type-Driven API Design in Rust (video)" by Will Crichton</a>: walks through designing an API in Rust step-by-step, showing how Rust's type system can be leveraged to guard against increasingly more kinds of invalid uses.</li> "Async destructors, async genericity and completion futures" by Sabrina Jewson</a>: a foray into the design challenges of designing an "async Drop" feature in Rust, providing some sharp insights on the matter. Deep-dives into the design problem space as done at the start of this post are in my opinion something we need more of in the Rust project.</li> "The Tower of Weakenings: Memory Models for Everyone" by Aria Beingessner</a>: Explains what provenance is, why we should care, and how we can bring strict provenance to Rust. This is one of those posts which really peels back the cover on the sticks-and-bubblegum nature we're building on, and then proposes a way to actually fix that without proposing everyone need become an expert in the process.</li> </ul> Capabilities</h3> "No Ghosts!" by Sunfishcode</a>: proposes and explores a design principle for components in complex software systems.</li> "What is a Capability" by Sunfishcode</a>: Answers the question: "What is a capability?".</li> "Capabilities: effects for free" by Craig et al.</a>: bridges the concept of "capabilities" with "effects", showing how we can use effects as capabilities.</li> "Capabilities for Resources and Effects" by Martin Odersky</a>: The Scala project got a 7-year EU research grant to investigate adding capabilities and effects into the language. This describes what that work will be focusing on.</li> "Contexts and Capabilities in Rust" by Tyler Mandry</a>: (I was involved with this), describes the "context problem" in Rust, and explains how we could solve this by adding a new "with-clause" language feature.</li> "The Path to Components (video)" by Luke Wagner</a>: Goes in-depth on the the WASM component model: what it is, where it's currently at, and where it's headed.</li> </ul> Systemic Safety</h3> "Blaming human error for an outage is like blaming gravity for a building falling over" — Tef on Twitter</a></p> </blockquote> "The New View of Safety (video)" by Todd Conklin</a>: "The next time something goes wrong, switch you thinking from 'who failed' to 'what failed'." Incredibly entertaining speaker talking at the United Steel Workers conference on health safety and environment. It's not directly about tech, but highly applicable to tech.</li> "The Field Guide to Understanding Human Error" by Sidney Dekker</a>: I've started this and plan to finish it over the holidays. But so far it's proben to be an incredibly useful resource reflecting on what failure is, and how to think about it.</li> "How Complex Systems Fail" by Richard Cook</a>: A great overview of how to think about failure, linking to relevant research backing up each point. Authored by a medical doctor, but highly relevant to many fields including software engineering.</li> "Confessions of a Recovering Engineer: Transportation for a Strong Town"</a>: An interesting dive into the world of transportation, how to think about safety, and provides a perspective on the politics of institutionalized engineering.</li> "Crash-Only Software" by Candea et al.</a>: Another classic computer science paper which I'd only ever heard about, but never actually read. It provides interesting perspectives on failure modes in computing, and how to deal with them.</li> </ul> Compilers</h3> "Cranelift Progress in 2022" by Chris Fallin</a>: As someone who likes compiler theory more than they enjoy working on compilers directly, this post is a dream. It covers all the compiler improvements that have happened on Cranelift over the past year. It's super inspiring to see a production-grade project have such a strong focus on correctness and meaningfully better abstractions. Being able to follow this work in the open is such a treat!</li> "A catalogue of optimizing transformations" by Allen et al.</a>: I've seen this paper referenced many times, but I didn't actually read it until this year. It's pretty pragmatic in how it covers the main optimizations a compiler can implement. Folks sometimes refer to this as: "the good ones", as just by implementing these you can get like 80% of the way there.</li> </ul> Miscellaneous</h3> "What we owe each other" by Vagrant Gautam</a>: A really good post on justice, feelings, and accountability. Being a human can be hard sometimes, and Vagrant does a fantastic job navigating this. Not an easy read, but one that I found worthwhile.</li> "The Five Minute Feedback Fix" by Hillel Wayne</a>: shares a practical formal verification technique which takes all of five minutes to learn, and can be applied to virtually any language or project.</li> "Bullshit Jobs" by David Graeber</a>: A reflection on the nature of work, which felt relevant to me since, y'know, I'm a worker and all.</li> "IETF RFC 9110: HTTP Semantics" by Fielding et al.</a>: The IETF published a new series of HTTP specs this year, unifying the myriad of previous documents into "semantics", "caching", and separate documents for HTTP/1.1, HTTP/2, and HTTP/3. The "semantics" document is fantastic if you just want to learn how HTTP works, without going into the exact details of the various backing wire protocols. I'm a big fan!</li> </ul> Conclusion</h2> I hope some of this was interesting. I'm off to finish reading some of the books on this list. Happy holidays!</p> Rust 2023 (by Yosh) Wed, 21 Dec 2022 00:00:00 +0000 Previous posts1</a></sup>: 2018</a>, 2019</a>, 2020</a>, 2021</a></em></p> ^{1</sup> I didn't write a "Rust 2022" post, sorry.</p> </div>}The core team used to put out a yearly call for blog posts. My colleage Nick published their "Rust in 2023"</a> post last week, and encouraged others to do the same. I like the idea of taking a moment to reflect on larger topics, and so well, why not write a post!</p> I want to do this a bit differently from the usual formula though. Rather than writing something with the specific intent to build some sort of "Rust 2023 roadmap", I want to instead take this as an opportunity to reflect on the state, values, and priorities of the Rust project. More of a snapshot of my current perspectives, than a concrete list of action items I think should be tackled. Here goes!</p> Async</h2> Probably most of my work will continue to focus on async Rust. A majority of programming jobs are either frontend or backend related 2</a></sup>, and async is important to both. Anecdotally I know that async has been a major driver for the adoption of Rust - and the more we can improve async Rust, likely the better we'll do in those fields.</p> ^2</sup>The 2022 stackoverflow survey notes 26% of people work on front-end, and 43% of people do backend work. These numbers are non-exclusive, since folks could be doing both (src</a>) - but I think it's fair to categorize this as a majority of jobs.</p> </div> Work has been progressing a lot here. Diagnostics have become significantly better. We now have IntoFuture</code> stabilized. And both safe stack-pinning and async fns in traits are on the horizon. And slightly further out is the work on async iteration, async drop, async main, and keyword generics.</p> I should probably get around to writing the end-of-year update post for the Async WG I promised I would write. But the tldr is that async Rust is important to Rust's future, we've made a lot of progress over the past year - and I think we're in a pretty good spot to keep making progress.</p> Ambition</h2> It's winter holidays, and I'm not going to ping people to clear quotes. But someone I respect a lot once remarked: "Rust does best when we're ambitious"</em> - and I agree strongly with this. I think the work we're doing on Rust is far from done. Rust is in a spot where people can be productive, but there are still so many limitations. And when people run into this, I want us to do better than: "Well, that's just the way things are". We can in fact have nice things!</p> As a general attitude, I want us to stay ambitious. To imagine a world in which better things are in fact possible. And then doing the work to get there. Rust combines the desire for a radically better programming language with the motivation to actually get things in people's hands. And I think our work is some ways only just getting started 3</a></sup>.</p> ^{3</sup> It's hard to say how long programming languages are expected to last, but we're seeing Rust adopted in some pretty high-stakes places which are likely to stay around for a very long time</em>. This means it's possible Rust will outlast us, meaning that in retrospect now might very well still be "the early days" in terms of Rust adoption. We won't know for sure until we're there, but imo this is not an unlikely outcome of our current trajectory.</p> </div>}^{4</sup> I feel like everyone attributes this quote to someone else at this point, but: the design space of a programming language is fractal! There are infinite details to dig into, and there's also a real art in deciding what to prioritize.</p> </div>}Cohesion</h2> "What is Rust" is one of those fun open-ended questions which doesn't have a right answer. After all: Rust is a programming language. Or maybe Rust is a feeling in our hearts? I think Rust is the friends we make along the way. Really, it's us who get to decide what Rust means to us.</p> Pragmatically, I think Rust is not just</em> a programming language. Cargo is Rust. Rustup is Rust. Crates.io is Rust. All the tools, guides, websites, people, and programs together make "Rust". And I think that with that come some interesting questions about governance, power, and cohesion as well.</p> I won't bore you with some long essay on the reflection of power dynamics within open source ecosystems. But what I will say is that Rust has done remarkably well in ensuring people using the language are presented with a relatively cohesive experience! The compiler, formatter, versioning tool, guide, package manager, and standard library are all owned and operated by the Rust project. And I think this is a strength we should continue to build on.</p> I'm really happy that we've recently seen the addition of the cargo add</code> command</a>, removing the need to rely on the external (yet trusted) cargo-edit</code> crate. And if I'm not mistaken work on cargo-semver-checks</a> is currently ongoing with the intent to integrate it into mainline cargo as well. 5</a></sup></p> ^5</sup>Somewhat proudly one of my own contributions along these lines in the past year has been to bring the core functionality of the num_cpus</code> crate into the stdlib</a>. One less crate to learn about!</p> </div> I believe that one of the main strengths of Rust is that we're creating something together which looks and feels cohesive. I think it's good to keep adding features as long they end up feeling part of a whole. And I think this is something that's worth reflecting on, and identifying potential places where we may be able to lean further into this.</p> Automation</h2> I sometimes think about the 80/20 rule: "80% of work takes 20% of time; the remaining 20% in 80% of time." I'm not sure if this has a proper name, but there's also what I like to call the 100% rule: "Once you've solved 100% of a problem, you can stop thinking about it completely".</em></p> Take for example memory safety in systems programming languages. Using fuzzers and sanitizers you may be able to easily address 80% of bugs. But the remaining 20% of bugs will take some effort to find. And even when you do; code changes over time, which means that you'll be unlikely to ever afford to stop looking for these types of bugs.</p> Compare that with the borrow checker, which guarantees there will never</em> be any memory safety bugs: meaning you can catagorically eliminate the concern alltogether. The 80/20 rule is often used to indicate that "80% of effort is good enough". But I think it's important to realize that sometimes achieving 100% of a solution can provide a value-add which far outweighs all incremental steps that come before it.</p> I want us to keep looking for ways in which we can structurally eliminate issues through automation, so we can stop thinking about them alltogether. Taking cargo-semver-checks</a> as an example again: rather than needing to learn and check for things like "variance" and "contravariance"</a>. When publishing new versions of crates, we should just have a tool which checks it for us - and patiently explains how we can fix the issues, even if we're not completely sure about what the underlying rules are.</p> I think it was Ralf Jung who once said something along the lines of: "The job of an expert is to learn everything about a field there is to learn, and then distill it so that others don't have to."</em> 6</a></sup>. And in my opinion building tools is one of the best ways we have to propagate domain expertise. I think this is something which we should keep leaning into 7</a></sup>.</p> ^{6</sup> This was definitely a TWIR quote of the week at some point. I think late 2020/early 2021 ish? If someone remembers the link I'll update this post!</p> </div>}^7</sup>I believe programming language features such as the borrow checker, or capabilities</a> fall under this as well. Rather than memorizing and manually validating a set of rules, we can/should use automation (a compiler) to automate this reasoning for us. Because for example even interpreted languages are not free from having to reason about lifetimes and ownership; the only difference is that the language doesn't provide facilities to automate this reasoning. So people instead have to rely on techniques such as defensive copying to prevent aliasing bugs from occurring. (This could probably be a post on its own, so I'll just stop here hah.)</p> </div> Conclusion</h2> I can probably keep talking about Rust at length; that's what this blog is for after all. But I think these are some of the key points I wanted to hit. Before closing out though, I probably should briefly cover my thoughts on governance:</p> I'm happy folks are working on governance. I trust them and their work.</li> I'm happy we're seeing different variants of "light weight RFCs" popping up. I believe both compiler (MCP</a>) and libs (ACP</a>) have these now, and I'm excited to give some of these a shot.</li> Governance is fundamentally about relationships, and as long as the project changes, so will governance. The work here will likely never be "done", and that's probably just the nature of it.</li> </ul> To close out: I think we're doing great, and I'm excited for the direction we're headed. Happy 2023!</p> Async Iteration II: The Async Iterator Crate Mon, 19 Dec 2022 00:00:00 +0000 Async Iteration I: Semantics</a></li> Async Iteration II: Async Iterator Crate</a> (this post)</li> Async Iteration III: Async Iterator Trait</a></li> </ol> Hey all, today was my first day back from PTO. While away, I was thinking what I often think: I should write more on here. With Twitter effectively defunct, and my Mastodon being set to ephemeral mode (posts auto-delete after 30 days), I should start keeping a better record of what I'm up to somewhere more permanent</em>. And this blog seems as a good a place as any.</p> Async Iterator</h2> I wanted to share that I've published a crate: async-iterator</code></a>. Its purpose was partly to test out the new "async fn in traits"</a> feature on nightly. And partly to have a workable sketch of what we expect "async iterator" to behave like.</p> use </span>async_iterator::prelude::; </span> </span>async </span>fn </span>foo</span>(</span>iter</span>: impl Iterator<Item = </span>u32</span>>) -> Vec<</span>u32</span>> { </span> iter.</span>collect</span>().await </span>} </span></code></pre> This is not final</em> shape of the trait though, since we're working on keyword generics</a> and expect to retroactively add the async Iterator</code> functionality to the Iterator</code> trait which then becomes generic over "asyncness". But those should just be the same semantics exposed through a different mechanism. Other than that, the crate is pretty minimal, yet covers pretty much all the common usage patterns:</p> The base async Iterator</code> trait implemented through async fn next</code> instead of fn poll_next</code></li> The ability to collect</code> into a vec</code>, using actual async IntoIterator</code> and FromIterator</code> bounds</li> The ability to async map</code> over values in the iterator</li> The ability to extend</code> vec</code> with an async iterator</li> </ul> The only thing missing is async fn filter</code>, which borrows items and so needs some form of "async closures" to work. But all in all this means it works! - "async fn in traits" is real on nightly, and actually does what we want it to do! I think that's really exciting! I don't think the async-iterator</code> crate should be used in actual projects though. It's an interface which only works on nightly, and interfaces only really work when they're uniformly adopted. And between futures 0.3.0 and the stdlib I don't think we should migrate to any in-between interfaces in the interim. But I do think it will be really useful to write experiments against!</p> Future Directions</h2> I still have to check in with the "async fns in traits" team to see what they're up to (being gone for a month is a long time). But something I might consider doing in the new year is do a rewrite of the async-h1 crate</a> based on "async fn in traits". The internals are basically a giant poll</code>-based state machine, and we've theorized for a while now that rewriting it in terms of async fn</code>s would significantly simplify its internals. It seems like this should finally be possible, but I'll have to check with folks first whether doing this will actually be helpful now as well.</p> Keywords II: Const Syntax Wed, 26 Oct 2022 00:00:00 +0000 On the Keyword Generics Initiative</a> we spend a lot of time talking about, well, keyword generics! We're working on designing a system which works with Rust's various effects</em>, which are represented through keywords. And sometimes we discover interesting interactions or meanings of keywords.</p> Previously I've written about the meaning of unsafe</code></a>, showing how it actually is a pair of keywords whose meaning depends on whether it's in caller or definition position. const</code> shares some similarities to this, and in this post we'll be looking closer at the different uses of const</code>.</p> always-const execution</h2> If you want to execute a function during compilation, you need to have a way to make that happen. This is what we've been referring to as an "always const" execution, where you have a context which will always be evaluated at compile-time and never during runtime.</p> In Rust we can create an always-const context in a few different ways:</p> // Defining a `const` value </span>const </span>HELLO</span>: &</span>str </span>= { </span> "</span>hello</span>" </span>}; </span> </span>// Defining a const value in an array expression </span>[</span>0</span>u8</span>; {</span>1 </span>+ </span>1</span>}]; </span> </span>// Using `RFC 2920 experimental in-line const` </span>let</span> x = </span>const </span>{ "</span>hello</span>" }; </span> </span>// In const-generic parameters through `#![feature(`const_evaluatable_checked`)]` </span>fn </span>foo</span><</span>const</span> N: </span>usize</span>>() -> [</span>u8</span>; N + </span>1</span>] { [</span>0</span>; N + </span>1</span>} } </span></code></pre> These contexts also disallow</em> the use of non-const values. Using them there will quickly yield an error:</p> fn </span>foo</span>(</span>length</span>: </span>usize</span>) { </span> [</span>0</span>u8</span>; length]; </span>} </span></code></pre> Compiling playground v0.0.1 (/playground) </span>error[E0435]: attempt to use a non-constant value in a constant </span> --> src/main.rs:5:11 </span> \| </span>4 \| fn foo(length: usize) { </span> \| ------ this would need to be a `const` </span>5 \| [0u8; length]; </span> \| ^^^^^^ </span> </span>For more information about this error, try `rustc --explain E0435`. </span>error: could not compile `playground` due to previous error </span></code></pre> never-const execution</h2> Code which cannot be evaluated during compilation is considered "never-const". These contexts are effectively "runtime-only". Because const</code> is opt-in in today's rust, it means that functions by default will not and cannot be evaluated during compilation.</p> fn </span>hello</span>() -> &</span>'static str </span>{ </span> "</span>hello</span>" </span>} </span> </span>const </span>HELLO</span>: &</span>str </span>= </span>hello</span>(); </span></code></pre> error[E0015]: cannot call non-const fn `hello` in constants </span> --> src/lib.rs:5:23 </span> \| </span>5 \| const HELLO: &str = hello(); </span> \| ^^^^^^^ </span> \| </span> = note: calls in constants are limited to constant functions, tuple structs and tuple variants </span></code></pre> maybe-const execution</h2> The third type of const</code> context is "maybe-const". This is a context which can be evaluated both during compilation and at runtime. The classic const fn</code> definition is considered a "maybe-const" context:</p> // Can be evaluated both at runtime and during `const` execution. </span>const fn </span>square</span>(</span>num</span>: </span>u64</span>) -> </span>u64 </span>{ </span> num * num </span>} </span> </span>// Const evaluation </span>const </span>NUM</span>: </span>u64 </span>= </span>square</span>(</span>12</span>); </span> </span>// Runtime evaluation </span>fn </span>main</span>() { </span> println!("</span>{}</span>", </span>square</span>(</span>12</span>)); </span>} </span></code></pre> As an aside: In practice both these functions will be evaluated during compilation thanks to an optimization called "const folding". But that's just an optimization; not all code can be const-folded, and the code wouldn't compile if it wasn't valid at runtime. Whether or not a function is marked const</code> has no impact on whether or not the code will be const-folded. But I figured it'd at least be worth mentioning the optimization as existing once in this post.</p> const traits</h2> Something which we don't yet have is the idea of "const traits". Right now you can think of them like this:</p> const trait </span>Trait { </span> </span>fn </span>func</span>() -> </span>usize</span>; </span>} </span></code></pre> This is a "const trait" with a single "const method" which returns a type. Just like const fn</code>, this should be considered a "maybe-const context": it should both be able to be evaluated at runtime and during compilation. For example:</p> // evaluates both at runtime and during compilation </span>const fn </span>foo</span><T: </span>~</span>const</span> Trait>() -> </span>usize </span>{ </span> T::func() </span>} </span></code></pre> the need for always-const bounds</h2> The current system of const</code> syntax has some interesting limitations. Take for example the following code:</p> // Can be called at runtime with a `T` which doesn't impl </span>// `const Trait`. </span>const fn </span>foo</span><T: </span>~</span>const</span> Trait>() { </span> [(); <T as Trait>::func()]; </span> </span>// ^ illegal </span>} </span></code></pre> What we're seeing here is an "always-const" context nested in a "maybe-const" context. A fun twist on the usual effect sandwich. In theory that should be fine, but the issue here is that we're using a variable which is "maybe-const". To get an understanding of why this is bad, here is roughly the same example, but without the traits:</p> const fn </span>foo</span>(</span>len</span>: </span>usize</span>) { </span> [(); len]; </span>} </span></code></pre> This yields the following error about wanting to have an "always const" value, but what we got is a "maybe const" value:</p> Compiling playground v0.0.1 (/playground) </span>error[E0435]: attempt to use a non-constant value in a constant </span> --> src/lib.rs:2:9 </span> \| </span>1 \| const fn foo(len: usize) { </span> \| --- this would need to be a `const` </span>2 \| [(); len]; </span> \| ^^^ </span> </span>For more information about this error, try `rustc --explain E0435`. </span></code></pre> The earlier trait example this would yield a very similar error, but using more steps. Thinking about solutions here, we practically have two ways to design our way out of this:</p> Find a way to declare the maybe-const foo</code> context as always-const.</li> Find a way to declare the maybe-const T: ~const Trait</code> as always-const.</li> </ol> This is an unresolved problem, so we're full on speculating here. But we could imagine that if we support ~const Trait</code> meaning "maybe-const", we could also support const Trait</code> to mean "always const":</p> // This now compiles because `T` is always guaranteed to be `const`. </span>const fn </span>foo</span><T: </span>const</span> Trait>() { </span> [(); <T as Trait>::func()]; </span>} </span></code></pre> This gets us into a difficult spot though. This would create a difference between the meaning of const</code> when declared and when used in parameters. Ideally we could, like, not</em> have that. Because as we've seen in other places having the same word mean different things in different contexts can be pretty confusing.</p> The need to specialize</h2> Another thing to mention here is the idea of specialization</em>. In the compiler we have const_eval_select</code></a> which allows you to specialize depending on whether you're executing at runtime or during compilation. This can be useful to optimize implementations with, since more assumptions can be made about the exact environment you'rer running on during runtime. During compilation Rust is executed in an interpreter, which by design creates a uniform environment regardless of what platform the compiler is actually running on.</p> #![</span>feature</span>(const_eval_select)] </span>#![</span>feature</span>(core_intrinsics)] </span>use </span>std::intrinsics::const_eval_select; </span> </span>// A function which returns a different result based on runtime or compiletime </span>pub const fn </span>inconsistent</span>() -> </span>i32 </span>{ </span> </span>fn </span>runtime</span>() -> </span>i32 </span>{ </span>1 </span>} </span> </span>const fn </span>compiletime</span>() -> </span>i32 </span>{ </span>2 </span>} </span> </span>unsafe </span>{ </span> </span>const_eval_select</span>((), compiletime, runtime) </span> } </span>} </span> </span>const </span>CONST_X</span>: </span>i32 </span>= </span>inconsistent</span>(); </span>let</span> x = </span>inconsistent</span>(); </span>assert!(x != </span>CONST_X</span>); </span></code></pre> Something we're talking about in the initiative group is: "How can we enable people to author different versions of the same resource which only differentiate based on the effect?" This would effectively fill the same role as const_eval_select</code>, but you could imagine it working for other effects such as async</code> as well. The syntax we've been playing around is some version of:</p> // maybe-async definition using an intrinsic to switch based on the context </span>async<A> </span>fn </span>foo</span>() { </span>async_eval_select</span>(..) } </span> </span>// the equivalent overload-based variant </span>async </span>fn </span>foo</span>() -> </span>usize </span>{ </span>1 </span>} </span>// always async </span>fn </span>foo</span>() -> </span>usize </span>{ </span>2 </span>} </span>// never async </span></code></pre> Something we haven't really figured out is how we should define this for const</code> definitions. We'd want to declare two variants: one which is definitely not const. And one which definitely is. And have these create a unified definition.</p> // maybe-const variant with the intrinsic </span>const fn </span>foo</span>() -> </span>usize </span>{ </span>const_eval_select</span>(..) } </span> </span>// this wouldn't work because we can't define an "always const" context </span>const fn </span>foo</span>() -> </span>usize </span>{} </span>// sometimes const </span>fn </span>foo</span>() -> </span>usize </span>{} </span>// never const </span></code></pre> Summary: what is const?</h2> Let's recap everything we've seen about const</code> so far. In today's Rust the const</code> effect can roughly be thought of as having three different modes, split between declaration and usage:</p> </th> declaration</th> usage</th></tr></thead> keyword never applies</strong></td> fn foo() {}</code></td> fn bar() { foo() }</code></td></tr> keyword always applies</strong></td> -</td> const FOO: () = {};</code> †</em></td></tr> keyword conditionally applies</strong></td> const fn foo() {}</code></td> const fn bar() { foo() }</code></td></tr> </tbody></table> †: an "always const" context can only ever call "maybe-const" declarations since there is no "always-const" declaration available in Rust today.</em></li> </ul> The const</code> keyword here means different things. const FOO</code> is used to declare an "always-const" context. But when you write const fn foo</code> you're in fact declaring a "sometimes-const" context. This means const</code> does not have a consistent meaning in the language: depending on where it's used, it may either signal an "always const" context, or a "maybe const" context.</strong> And as we've seen in the case of trait bounds, there is a meaningful observable distinction between "always const" and "maybe const" contexts, bounds, and declarations.</p> This following table shows how "const definitions" interact with "const contexts". A "never const" context is runtime-only, an "always const" context is compilation-only, and a "sometimes const" context can be evaluated both during compilation and at runtime:</p> Can [def] be evaluated in [context]?</th> never const context</th> sometimes const context</th> always const context</th></tr></thead> never const def</strong></td> ✅</td> ❌</td> ❌</td></tr> sometimes const def</strong></td> ✅</td> ✅</td> ✅</td></tr> always const def</strong></td> ❌</td> ❌</td> ✅</td></tr> </tbody></table> The way to think about "const" is as a subset of "base" Rust. We gain the ability to evaluate code during compilation by removing capabilities from the language. This means we don't get the ability to access statics, or any host functionality. Because "const Rust" is a subset of "base Rust", you can also think about it in an inverse and consider "base Rust" a superset of "const Rust". If we pull in "async Rust" as well, we can plot the following diagram:</p> +---------------------------+ </span> \| +-----------------------+ \| Compute values: </span> \| \| +-------------------+ \| \| - types </span> \| \| \| \| \| \| - numbers </span> \| \| \| const Rust \|-------{ - functions </span> \| \| \| \| \| \| - control flow </span> Host access: \| \| +-------------------+ \| \| - traits (planned) </span> - networking \| \| \| \| - containers (planned) </span> - filesystem }-----\| "base" Rust \| \| </span> - threads \| \| \| \| </span> - system time \| +-----------------------+ \| </span> - statics \| \| Control over execution: </span> \| async Rust \|---{ - ad-hoc concurrency </span> \| \| - ad-hoc cancellation/pausing/resumption </span> +---------------------------+ - higher-order control flow; ad-hoc timeouts, etc. </span></code></pre> Back to the drawing board</h2> Okay, analysis time stops here. What I've said so far is mostly factual, and I think we should be able to reference as mostly true for the foreseeable future. From this point onward we put our imagination caps on and strap in to speculate about what could be</em>. If we could go back to the drawing board, how could we change things?</p> Well, for one, we now know we not only want to have "maybe const" functions and types. We also want "maybe async", "maybe throws"; basically "maybe anything</em>" effects. The fact that const fn</code> is "maybe" by default but async</code> isn't can be a bit surprising. Here they are side-by-side:</p> </th> keyword async</code></th> keyword const</code></th></tr></thead> keyword never applies</strong></td> fn foo() {}</code></td> fn foo() {}</code></td></tr> keyword always applies</strong></td> async fn foo() {}</code></td> const FOO: () = {};</code>†</em></td></tr> keyword conditionally applies</strong></td> async<A> fn foo() {}</code> ‡</em></td> const fn foo() {}</code></td></tr> </tbody></table> †: "always-const" functions don't exist in Rust today, so an alternate method is shown here</em></li> ‡ : placeholder syntax for functions where the keyword conditionally applies</em></li> </ul> The difference is pretty striking right? We don't have an "always const" function. And what we mean by "maybe const" in one place means "always async" in the other. When we look at usage</em>, const</code> and async</code> look a lot more similar though:</p> // Using `RFC 2920 experimental in-line const` </span>let</span> value = </span>const </span>{ "</span>hello</span>" }; </span> </span>// async block </span>let</span> value = async { "</span>hello</span>" }.await; </span></code></pre> They're not quite</em> the same because they're not the same feature. But the usage here feels pretty good already. Now how could we bring the rest of const</code> to feel a bit more in line with the other keywords. Probably the boldest answer would be to change the meaning of const fn</code> to mean: "always const". It could then use a similar syntax to "maybe async" to declare "maybe const" contexts, yielding us the following table:</p> </th> keyword async</code></th> keyword const</code></th></tr></thead> keyword never applies</strong></td> fn foo() {}</code></td> fn foo() {}</code></td></tr> keyword always applies</strong></td> async fn foo() {}</code></td> const fn foo() {}</code> †</em></td></tr> keyword conditionally applies</strong></td> async<A> fn foo() {}</code> ‡</em></td> const<C> fn foo() {}</code> ‡</em></td></tr> </tbody></table> † : "always-const" functions are new in this table, and use the existing "maybe-const" definition</em></li> ‡ : placeholder syntax for functions where the keyword conditionally applies</em></li> </ul> This would fix the main inconsistency between the effects, enabling them to be used in a much more uniform manner. Looking at our earlier example, the original formulation would have to become:</p> // declraring a "maybe const" context which takes "maybe const" params </span>const</span><C> </span>fn </span>foo</span><T: </span>const</span><C> Trait>() { </span> [(); <T as </span>const</span> Trait>::func()]; </span> </span>// ^ illegal; `T` must be "always const" </span>} </span></code></pre> This, at least to me, feels like it more clearly describes the problem at hand. const fn</code> and T: const</code> now use the same syntax to indicate they're conditional. And we're trying to cast a T</code> which is "maybe const" to be "always const". Diagnostics should make this fairly easy to point out.</p> This means the solutions to this issue could also be a lot clearer: we either cast T</code> to be "always const", or mark the const fn foo</code> to be an "always const context":</p> // maybe-const fn foo, with an always-const trait T </span>const</span><C> </span>fn </span>foo</span><T: </span>const</span> Trait>() { </span> [(); <T as </span>const</span> Trait>::func()]; </span>} </span> </span>// always-const fn foo, with an always-const trait T </span>const fn </span>foo</span><T: Trait>() { </span> [(); <T as Trait>::func()]; </span>} </span></code></pre> I'm less sure about having const fn foo</code> imply that all generics are also const</code>, all casts are const</code>, etc. Maybe they should be specified as const</code> as well? But on the other hand: they would have to be const</code> for it to work, so maybe it can be ommitted? Though that's mostly a syntactical question, as it wouldn't affect the semantics.</p> Making the meaning of const</code> unambiguous seems like it would make cases like these a lot easier to work with. And it doesn't just stop at bounds; if we wanted to enable effect-based overloading, the example we saw earlier would Just Work:</p> // maybe-const variant with the intrinsic </span>const</span><C> </span>fn </span>foo</span>() -> </span>usize </span>{ </span>const_eval_select</span>(..) } </span> </span>// under the new rules this would be equivalent to a single "maybe const" </span>// declaration: </span>const fn </span>foo</span>() -> </span>usize </span>{} </span>// always const </span>fn </span>foo</span>() -> </span>usize </span>{} </span>// never const </span></code></pre> If we were to introduce this change, I believe we could actually do it over an edition bound. The main "new" feature is the ability to declare "always-const" functions, traits, methods, etc. These couldn't be declared in older editions; but because changing an existing "maybe-const" definition to become "always-const" would be considered a breaking change, it means that it wouldn't just break existing code simply by existing.</p> const fn</code> code written on an older edition would be reinterpreted as "maybe-const" in newer editions. And possibly we could also just add support for trait bounds through const<C></code> on the old edition too, so on older editions there would just be multiple ways to declare similar "maybe const" contexts:</p> //! The different meanings of `const` between editions </span> </span>// old edition </span>const fn </span>foo</span>() {} </span>// maybe-const no traits </span>const</span><C> </span>fn </span>foo</span><T: </span>const</span><C> Foo>() {} </span>// maybe-const with traits </span> </span>// new edition </span>const fn </span>foo</span>() {} </span>// always const </span>const</span><C> </span>fn </span>foo</span>() {} </span>// maybe-const no traits </span>const</span><C> </span>fn </span>foo</span><T: </span>const</span><C> Foo>() {} </span>// maybe-const with traits </span></code></pre> //! Even if older editions can't define always-const functions, </span>//! they could still be available to older editions. The hardest part </span>//! is probably figuring out the diagnostics for the older edition to talk </span>//! about a concept which can be used, but not created. </span> </span>// new edition: defines an always-const function </span>crate</span> new { </span> </span>const fn </span>foo</span>() {} </span>// always const declaration </span>} </span> </span>// old edition; uses an always-const function </span>crate</span> old { </span> </span>use super</span>::new::foo; </span> </span>const</span> Foo: () = </span>foo</span>(); </span>// okay to call an always-const fn in an old edition </span> </span>// but no way to declare new ones </span>} </span></code></pre> As I've mentioned before: the exact syntax for conditional effects is still undecided. But regardless of what we end up with, the idea is that we can create something consistent between keywords.</p> Conclusion</h2> In this post we've gone over what const</code> is, how it relates to other keywords, and gone deeper on the different uses and meanings of const</code> in Rust today. We've then taken a shot at trying to resolve these issues, with an outline of how we could potentially normalize the meaning of the const</code> keyword over an edition bound.</p> Thanks to Oli</a> for proof reading this post!</em></p> Inline Crates Tue, 25 Oct 2022 00:00:00 +0000 People sometimes jest that Rust is just ML dressed up to look like C++. And I don't think that's entirely off: Rust has many of the key features present in ML languages. We have the same kind of type system (Hindley-Milner</a>), we have sum types, and we have a module system which isn't directly tied to a module hierarchy. I want to talk a bit more about Rust's module system here.</p> In Rust we distinguish between "crates"</a> and "modules"</a>. To people just learning about Rust the distinction can be a bit confusing. But in practice it makes sense to have both. In this post we're going to take a look at Rust's module system, what the differences are, and how we could introduce some features to bring crates and modules closer together.</p> Disclaimer: In this post I'm sharing some ideas on language design that heavily tie into the current compiler architecture. I want to make it super clear that I'm not necessarily advocating we make these changes, and I'm especially not trying to say that this should be prioritized. I just wanted to have this written down so it can be referenced later. Because I think there's value in writing things down even if they're not fully formed, feasible, or quite the right time.</em></p> What are crates?</h2> A crate is a unit of code with strict</em> encapsulation rules: Crates are also a unit which can individually be published to and pulled from crates.io. In the compiler it represents a translation unit, and is the boundary of parallelization 1</a></sup>. We're only able to compile entire crates in parallel; not yet sub-components of crates. And it also affects the boundaries of re-compilations.</p> ^1</sup>This is only about rustc</code> specifically - the LLVM part of the compilation may parallelize individual crates. You can read more on this in the rustc dev guide</a> and the rustc book</a>.</p> </div> Crates themselves also have strict rules on disallowing cyclic dependencies 2</a></sup>, and of course the orphan rules</a> 3</a></sup>. A crate is always tied to a file hierarchy on disk, and needs an accompanying Cargo.toml</code> file. Matklad (of Rust-Analyzer fame) goes into detail on their blog</a> how crates affect compilation times, and how changing crate hierarchies can be used to significantly improve throughput.</p> ^{2</sup> Given a</code> and b</code>. a</code> can depend on b</code>, or b</code> can depend on a</code>. But they can't both depend on each other, because that would cause a cycle in the dependency graph.</p> </div>}^{3</sup> Orphan rules come down to: if you want to implement a trait for a type, you must either define the trait or the type in your crate. You're not allowed to implement a trait you haven't defined for a type you haven't defined.</p> </div>}Finally, some features such as procedural macros</a> can only be defined in crates which have a specific configuration that disallows any other type of code to be exported. Crates, more generally, also serve as a boundary of configuration</em>.</p> backyard </span>├── Cargo.lock </span>├── Cargo.toml </span>└── src </span> ├── garden </span> │ └── vegetables.rs </span> ├── garden.rs </span> └── lib.rs </span></code></pre> An example of a crate representation, copied from the Rust book</a>.</em></p> For the purpose of this post we'll not be talking on the Rust 2015-era concept of extern crate</code></a>.</p> What are modules?</h2> A module is a unit of code with loose</em> encapsulation rules: Cycles between modules are allowed, and a module does not necessarily correspond to a file hierarchy. Crates are a lightweight way of splitting code up into logical components. Here's an example of a valid mod</code> relationship (playground</a>):</p> // example of cycles in modules </span> </span>pub mod </span>foo { </span> </span>use super</span>::bar::; </span> </span>pub struct </span>First; </span>} </span> </span>pub mod </span>bar { </span> </span>use super</span>::foo::; </span> </span>pub struct </span>Second; </span>} </span></code></pre> But modules can also correspond to a file hierarchy. In the following example, backyard</code> is the crate, with lib.rs</code> representing the entry point, and internally contains crate::garden</code>, and crate::garden::vegetables</code> as sub-modules:</p> backyard </span>├── Cargo.lock </span>├── Cargo.toml </span>└── src </span> ├── garden </span> │ └── vegetables.rs </span> ├── garden.rs </span> └── lib.rs </span></code></pre> Modules also accept visibility modifiers such as pub(crate)</code> or pub(super)</code>. This restricts the encapsulation rules somewhat by making the code accessible from fewer sites. It also changes things like being able to access fields on structs. But while it's stricter</em>, it still allows cycles between modules. Which is not the same level of guarantees which crates provide.</p> comparing modules and crates</h2> To summarize what we've covered so far:</p> name</th> encapsulation</th> file repr</th> in-source repr</th> cost to define and maintain</th></tr></thead> crate</code></td> strict</td> ✅</td> ❌</td> high</td></tr> mod</code></td> loose</td> ✅</td> ✅</td> low</td></tr> </tbody></table> Crates and modules are both incredibly useful concepts, with benefits and tradeoffs. But today it's significantly harder to define new crates, than it is to define new modules. Features like "workspace inheritence"</a> lower the barrier somewhat, but there's still a pretty large difference between how easy they are to introduce to your project.</p> Adding a new module is basically just adding a mod {}</code> and copying over the right imports - something which Rust-Analyzer can even do for you. But adding a new crate is far more involved: it requires creating a new module hierarchy, adding the right dependencies, adding the right imports to the workspace, and then linking the right imports back to the crates you want to use. This could be automated as well, but that only makes authoring crates easier - it doesn't help with readability.</p> And that's not even taking into account releasing</em> crates. When you add a new mod</code> to a crate it doesn't affect the ability to publish new crates. But when you create a new crate, you now have a new dependency. Which must be accurately versioned 4</a></sup> and published before the main crate can be released. The fact that "strict encapsulation semantics" are necessarily tied to an on-disk hierarchy is not ideal.</p> ^{4</sup> "workspace inheritence" can also help with versioning here.</p> </div>}in-line modules</h2> You can probably see where this is going. In my opinion Rust would benefit from detaching crates from the module hierarchy, and allowing them to also be defined in a manner similar to modules. A single "crate" should be able to contain many sub-crates, all of which enforce the same strict encapsulation rules as their on-disk counterparts. The only difference being in how they're declared. Much like workspace.inherit</code>, sub-crates would inherit the dependencies and version number of their parent crate. And unlike on-disk crates, they wouldn't be individually publishable to crates.io, but instead just be sent along with their parent crates.</p> To give an example of what this would look like; I'm thinking we adopt a very similar syntax to modules:</p> mod </span>foo {} </span>// define an in-line module </span>crate</span> foo {} </span>// define an in-line crate </span> </span>mod </span>bin; </span>// import a module from a file </span>crate</span> bin; </span>// import a crate from a file </span></code></pre> Example</h2> Being able to distinguish at the source-level between "public" and "private" crates is something which would be useful for larger projects in particular. An example of such a project is probably Rust-Analyzer, which I happen to be familiar with.</p> You can see an overview of all crates in the code map</a> section of the architecture.md</a> file. Rust-Analyzer clearly distinguishes between "API boundary"</a> crates, and all other crates. For example something like the ide</code> crate is a "boundary", but internally uses other crates such as ide-db</code>, and ide-assists</code> which are not boundaries:</p> crates/ </span> </span>hir-def/ </span> </span>hir-expand/ </span> </span>hir-ty/ </span> </span>hir/ </span># boundary </span> </span>ide-assists/ </span> </span>ide-completion/ </span> </span>ide-db/ </span> </span>ide-diagnostics/ </span> </span>ide-ssr/ </span> </span>ide/ </span># boundary </span></code></pre> I've contributed to Rust-Analyzer in the past, but I'm not on the RA team - so I can't speak with any authority about the project. But it's not unreasonable that a project similar to Rust-Analyzer could benefit from only surfacing the API boundary crates in the top-level crate hierarchy - without needing to give up the strict encapsulation rules Rust crates provide.</p> crates/ </span> </span>hir/ </span># boundary </span> </span>def/ </span> </span>expand/ </span> </span>ty/ </span> </span>ide/ </span># boundary </span> </span>assists/ </span> </span>completion/ </span> </span>db/ </span> </span>diagnostics/ </span> </span>ssr/ </span></code></pre> There may be projects who prefer to use a flat module hierarchy even if in-line crates become available. But when starting it seems easier to be able to quickly define a new crate, and only later move it to its own hierarchy. The point of the crate</code> keyword is to enable strict encapsulation without immediately having to resort to separate on-disk representations. Which should make it possible to let the structure found during prototyping hold all the way. As opposed of the current status quo where prototypes can be started using mod</code> statements, but eventually you'll want to refactor into separate crates.</p> Implementation</h2> I suspect implementing inline crates might not be an easy task. If I'm not mistaken 5</a></sup>, right now parallelism of compilation is driven by cargo</code> spawning a bunch of instances of rustc</code> which compile individual crates. If crates are represented as anything other than something you can point rustc(1)</code> to compile, it might need some rethinking of the architecture.</p> I don't think that's an argument against</em> inline crates though. But rather a reflection that the architecture of how we've organized the various rust compiler projects is reflected in the features provided it provides 6</a></sup>. And things features such as inline exist right at the seems of where the two projects meet.</p> ^{5</sup> Wesley checked this and said my understanding of this is indeed correct. I really wasn't sure about this in earlier drafts, but I'm thankful I was able to confirm ^^.</p> </div>}^6</sup>See: Conway's law</a>.</p> </div> Future Directions</h2> pub crate</h3> Alright, time to speculate a little. Conversations around "crates.io namespaces" pop up pretty regularly. The basic idea is that sometimes crates are correlated, and being able to group them together under a single namespace would be nice.</p> Take for example the windows</code> crate</a>. It's probably the single biggest crate on crates.io 7</a></sup>. Internally it's built up of lots of crates which are all versioned in lock-step, and hidden behind feature flags. Take for example win32</code>'s HTTP service API. To use it with the windows</code> crate you'd need to define it in your toml like so:</p> ^7</sup>I believe it exposes something like 30.000 unique types and traits. Like an order of magnitude more than e.g. web-sys</code></a> does, which covers the entire web platform</em>. windows</code> is a pretty good example for a "big" crate that needs to be broken up into sub-components for the compiler to even attempt to build it in reasonable time.</p> </div> [dependencies] </span>"</span>windows</span>" = { </span>version </span>= "</span>0.41.0</span>", </span>features </span>= ["</span>Win32_Networking_HttpServer</span>"] } </span></code></pre> Instead, if the windows</code> crate could be used as a namespace, we could imagine that instead of exclusively exposing APIs using feature flags, we may want to expose it as crates namespaced under windows</code>. For example, we could imagine something like this:</p> [dependencies] </span>"</span>windows/win32_networking_httpserver</span>" = "</span>0.41.0</span>" </span></code></pre> There are some big "ifs" involved here: we're assuming this can be neatly split out into a single crate. But let's assume we can. How would we split it out? Today that would mean creating a new hierarchy on disk, and putting the right code there. And then in turn resolving the dependency hierarchy ahead of time to correctly version and publish the dependencies in the right order.</p> What if we could have an 1:N</code> mapping to published crates from within crates? What if the definition inside the windows crate could look like this and it would be enough to both provide the feature from inside the windows</code> crate, and be publicly accessible as a sub-crate?:</p> #[</span>cfg</span>(feature = "</span>Win32_Networking_HttpServer</span>")] </span>pub crate</span> win32_networking_httpserver { ... } </span></code></pre> I'd imagine the dependencies and versions would all be shared from the Cargo.toml</code> definition. And dependencies between inline crates would be extrapolated and extracted into the manifest. I don't think this should replace</em> workspaces in any way. Workspaces are a great feature, and give a great deal of control. But it seems like it could be a way to lower the overall burden of authoring new packages.</p> WASI components</h3> And to continue the speculation: I've been wondering for a few months now whether we could provide an 1:N</code> mapping of Rust programs to WASI components</a>. In WebAssembly components serve as a boundary of strict encapsulation, not unlike but als not exactly along the lines of crates in Rust.</p> I'm fairly optimistic that WASI will present a best-in-class target to compile Rust to for networked services. And if we're able to provide fine-grained and easy-to-use mappings from Rust to WASI components, it might give Rust an edge over other languages. I suspect being able to lean into the strict encapsulation semantics of Rust crates may have meaningful benefits wrt compile targets that we're currently unable to leverage to its full potential 8</a></sup>.</p> ^{8</sup> I'm trying not to go full in on WASI encapsulation rules, and mapping Rust programs to WASI. But I think there's a lot there, and being able to define in-line crates may end up being a component in a full WASI story.</p> </div>}Conclusion</h2> That's about it I think. I would love to see more equivalence between "crates" and "modules" in Rust. Even if we one day see a loosening of the orphan rules, or we see Rust move to fine-grained paralellism - having in-line crates will still be useful, as they're easier to create and modify than their on-disk counterparts. Even if it's just to prototype and experiment before wanting to commit to a final design.</p> To come entirely clean: I don't see this work panning out anytime soon. It would require an effective rearchitecture of the compiler, and by itself this feature wouldn't carry its weight to justify doing that. But perhaps if enough reasons build up, eventually we'll look to make the changes which would also</em> enable this feature. Either way, I figured it'd be worth spelling this out.</p> I think we could ask some very interesting questions about what other benefits such a rearchitecture of the compiler could look like as well. Could compilation become more efficient? Could it become easier to optimize builds? What would we lose? What other features could be enabled? Even if we know we don't have the time for any of this now, I do think it's 1: fun to think about, and 2: perhaps something useful will come out of it.</p> I also want to at least state once that I don't believe this should be in place of any workspace improvements. But I see working in addition to</em> workspace improvements. I view both approaches as improvements to status quo, and I think they would end up complimenting each other rather well!</p> Thanks to Wesley Wiser</a> for proof reading!</em></p> Why Async Rust Mon, 26 Sep 2022 00:00:00 +0000 A lot of system design is about thinking of the nature of the domains we encounter. And only later, once we understand them, encoding this understanding in a way that machines can verify it.</p> I often find async Rust to be misunderstood. Conversations around "why async" often focus on performance 1</a></sup> - a topic which is highly dependent on workloads, and results with people wholly talking past each other. While performance is not a bad</em> reason to choose async Rust, we often we only notice performance when we experience a lack of it. So I want to instead on which features</em> async Rust provides which aren't present in non-async Rust. Though we'll talk a bit about performance too at the end of this post.</p> ^1</sup>The introduction to the Rust async book</a> summarizes the benefits of async Rust as follows: "In summary, asynchronous programming allows highly performant implementations that are suitable for low-level languages like Rust, while providing most of the ergonomic benefits of threads and coroutines."</p> </div> Language Hierarchy</h2> It's not uncommon to hear Rust and other languages described as "N languages in a trenchcoat". In Rust we have Rust's control flow constructs, we have the decl-macro meta language, we we have the trait system (which is turing-complete</a>), we have the cfg annotation language - and the list keeps going. But if we consider Rust as it is provided out of the box to us, as "base Rust" then there are some obvious modifiers to it:</p> unsafe</code> Rust: to use raw pointers and FFI</li> const</code> Rust: to compute values at compile-time</li> async</code> Rust: to enable non-blocking computation</li> </ul> All of these "modifier keyword" to the Rust language provide new capabilities which aren't present in "base Rust". But they may sometimes also take capabilities away. The way I've started thinking and talking about language features is in terms of "subset of the language" or "superset of the language". With that classification we can look at the modifier keywords again and make the following categorization:</p> unsafe</code> Rust: superset</li> const</code> Rust: subset</li> async</code> Rust: superset</li> </ul> unsafe</code> Rust only adds</em> the ability to use raw pointers. async</code> only adds the ability to .await</code> values. But const</code> adds</em> the ability to compute values during compilation, but removes</em> the ability to use statics and access things like the network or filesystem.</p> If language features only add</em> to base Rust then they're considered supersets. But if the features they add require they also restrict other features, then they're considered subsets. In the case of const</code>, all const</code> functions can be executed at runtime. But not all code which can be executed at runtime can be marked as const</code>.</p> The design of language features as sub/supersets of "base" Rust is essential: it ensures that the language keeps feeling cohesive</em>. And more so than size or scope, uniformity is what leads to the feeling of simplicity</a>.</p> async under the hood</h2> At its core async/.await</code> in Rust provides a standardized way to return types with a method on them that returns another type. Instead of returning a type directly, async</code> functions return an intermediate type first. 2</a></sup></p> ^{2</sup> Sure; "monad" 🙃</p> </div>}/// This function returns a string </span>fn </span>read_to_string</span>(</span>path</span>: Path) -> String { .. } </span> </span>/// This function returns a type which eventually returns a string </span>async </span>fn </span>read_to_string</span>(</span>path</span>: Path) -> String { .. } </span> </span>/// Instead of using `async fn` we can also write it like this. </span>/// `impl Future` here is a type-erased struct </span>fn </span>read_to_string</span>(</span>path</span>: Path) -> impl Future<Output = String> { .. } </span></code></pre> A future is just a type with a method on it (fn poll</code></a>). If we call the method at the right times and in the right way, then eventually it'll give us the equivalent of Option<T></code> where T</code> is the value that we wanted.</p> "A future is just a type with a method we can call" has a few implications. First of all, just like we can choose to call the method, we can also choose to not call the method. If we don't call the method at all then the future won't execute any work 3</a></sup>. Or we can choose to call it, and then stop</em> calling it for a while. Maybe we just call the method again later. We can even choose to drop the struct at any point, and then there's no more method to call and no more value to be obtained.</p> ^{3</sup> Yes, that's only the case for async fn / async {}</code>-futures. If you -> impl Future</code> you can do work before</em> constructing the future and returning it. But that's not considered a great pattern, and it's pretty rare in practice.</p> </div>}When we talk about "representing a computation in a type", we're actually talking about compiling down the async fn</code> and all of its .await</code> points into a state machine which knows how to suspend and resume from the various .await</code> points. These state machines are just structs with some fields in them, and and have an auto-generated Future::poll</code> implementation which knows how to correctly transition between the various states. To learn more about how these state machines work, I recommend watching "life of an async fn" by tmandry</a>.</p> The .await</code> syntax provides a way to ensure that none of the underlying poll</code> details surface in the user-syntax. Most usage of async/.await</code> looks just like non-async Rust, but with async/.await</code> annotations sprinkled on top.</p> Rust's Async Features</h2> The core feature async/.await</code> provides in Rust is control over execution</em>. Instead of the contract being:</p> > "function call" -> "output" </span></code></pre> We instead get access to an intermediate step:</p> > "function call" -> "computation" -> "output" </span></code></pre> The computation isn't just something which is hidden away from us anymore. With async/.await</code> we are empowered to manipulate the computation itself. This leads to several key capabilities:</p> The ability to suspend/cancel/pause/resume computation (ad-hoc cancellation)</li> The ability to execute computations concurrently (ad-hoc concurrency)</li> The ability to combine control over execution, cancellation, and concurrency</li> </ul> ad-hoc cancellation</h3> The ability to suspend/cancel/pause/resume any computation is incredibly useful. Out of the three being able to cancel execution is likely the most useful one. In both sync and async code it's both desireable to halt execution before it completes. But what's unique to async Rust is that any</em> computation can be halted in a uniform way. Every future can be cancelled, and all futures need to account for that 4</a></sup>.</strong></p> ^4</sup>Yes I'm fully aware of the concept of "cancellation-safety", and I have a post coming up discussing it in more detail. The tldr: "cancellation-safety" as a concept is under-specified, but importantly: "cancellation-safety" is only relevevant when using select! {}</code>, which is something which should not be used</a>. Correctly handling cancellation is something that manually authored futures still need to do though, and that can be tricky to do without "async Drop". But that's different from "cancellation-safety" or the idea that futures don't need to have to account for being cancelled at all.</p> </div> ad-hoc concurrency</h3> The ability to execute computations concurrently is another hallmark capability of async Rust. Any number of async fn</code>s can be run concurrently 5</a></sup>, and .await</code>ed together. In non-async Rust concurrency is typically tied to parallelism: many computations can be scheduled concurrently by using thread::spawn</code> and splitting it up that way. But async Rust separates concurrency from parallelism, providing more control 6</a></sup>. In non-async Rust concurrency and parallelism are interlinked, which among other things has performance implications. We'll talk more about the differences later in this post.</p> ^{5</sup> Bar any runtime faults such as deadlocks, which can occur if two computations are run concurrently but share a resource.</p> </div>}^{6</sup> Not all libraries make use of the separation between "concurrency" and "parallelism" though. We're still very much figuring out what async Rust even is, but many of the libraries in common use today don't necessarily surface this insight. I know many of my own older libraries sure don't.</p> </div>}combining cancellation and concurrency</h3> Now finally: when happens when you combine cancellation</em> and concurrency</em>? It allows us to do some interesting things! In my blog post "Async Time III: Cancellation and Signals"</a> I go in-depth on some of the things you can do with this. But the canonical example here is: timeouts. A timeout is a concurrent execution of some future and a timer future, mapped to a Result</code>:</p> If the future completes before the timer, we cancel the timer and return Ok</code></li> If the timer completes before the future, we cancel the future and return Err</code></li> </ul> That's cancellation + concurrency combined to provide a new third type of operation. To get a sense for why being able to time-out any computation is a useful property, I highly recommend reading Crash-Only Software by Candea and Fox</a> 7</a></sup>. But it doesn't just stop at timeouts: if we combine any of the suspend/cancel/pause/resume capabilities with concurrency, we unlock a myriad of new possible operations.</p> ^{7</sup> Shout out to Eric Holk for introducing me to this paper!</p> </div> These are the features async Rust enables. In non-async Rust concurrency, cancellation, and suspensions often require calling out to the underlying operating system - and it's not always supported. For example: Rust doesn't have a built-in way to cancel threads. The way to do it is usually to pass a channel to a thread, and periodically check it to see if some "cancel" message has been passed.</p> In contrast in async Rust any</em> computation can be paused, cancelled, or run concurrently. That doesn't mean that all computations should be run concurrently, or everything should have a timeout on it. But those decisions can be made on the basis of what we're implementing, rather than being limited by external factors such as system call availability.</p>}Performance: workloads</h2> When something is stated to perform better than something else, it's always worth asking: "under which circumstances?"</em> Performance is always dependent on the workload. In graphics cards benchmarks you'll often see differences between video cards based on which games are run. In CPU benchmarks it matters a lot whether a workload is primarily single-threaded or multi-threaded. And when we talk about software features "performance" is not a binary either, but highly dependent on the workload. When talking about parallel processing we can distinguish between two general categories of workloads:</p> throughput-oriented workloads</li> latency-oriented workloads</li> </ul> Thoughput-oriented workloads usually care about processing the maximum number of things in the shortest amount of time. While latency-oriented workloads care about processing each thing as quickly as possible. Sounds confusing? Let's make it more clear.</p> An example of software designed with throughput in mind is hadoop</a>. It's built for "offline" batch processing of workloads; where the most important design goal is to minimize the total CPU time spent to process data. When data is put into the system it may often take minutes or even hours to be processed. And that's fine. We don't care when</em> we get the results (within reason of course), we primarily care about using as few resources as possible to get the results.</p> Compare that to a public-facing HTTP server. Networking is typically latency-oriented</em>. We often care less about how many requests we can handle, than how quickly we can respond to them</em>. When a request comes in we don't want to take minutes or hours to generate a response. We want request-response roundtrips to be measured in at most milliseconds. And things like p99 tail-latencies</a> are often used as key performance indicators.</p> Async Rust is generally considered to be more latency-oriented than throughput-oriented. Runtimes such as async-std</code> and tokio</code> primarily care about keeping overall latency low, and preventing sudden latency-spikes.</p> Understanding what type of workload is being discussed is often the first step in discussing performance. A key benefit of async Rust is that most of the systems which use it have been tuned heavily to provide good performance for latency-oriented workloads - of which networking is an example. If you want to handle more throughput-oriented workloads, non-async crates like rayon</code></a> are often a better fit.</p> Performance: optimizations</h2> Async Rust separates concurrency</em> and parallelism</em> from each other. Sometimes the two are confused with each other, but they are in fact different:</p> parallelism is a resource, concurrency is a way of scheduling computation</p> </blockquote> It's best to think of "parallelism" as a maximum. For example, if your computer has two cores, the maximum amount of parallelism you have may be two 8</a></sup>. But parallelism is different from concurrency: computation can be interleaved on a single core, so while we're waiting on the network to perform work, we can run some other computations until we have a response. Even on single-threaded machines, computation can be interleaved and concurrent. And vice-versa: just because we've scheduled things in parallel doesn't mean computation is interleaved. No matter how many cores we're running on, if threads are taking turns by waiting on a single, shared lock, then the logical execution may in fact still happen sequentially.</p> ^8</sup>This is a simplified example; for a longer explainer see the std::thread::available_parallelism</a> docs.</p> </div> Let's take a look at an example of a concurrent workload in async and non-async Rust. In non-async Rust it's most common to use threads to achieve concurrent execution 9</a></sup>. But since threads are also the abstraction to achieve parallel</em> execution, it means that in non-async Rust concurrency</em> and parallelism</em> are often tightly interlinked.</p> In async Rust, we can separate concurrency from parallelism. If a workload is concurrent</em>, it doesn't imply it is also parallelizable</em> as well. This provides finer control over execution, which is the key strength of async Rust. Let's compare concurrency in non-async and async Rust:</p> ^9</sup>Unless you start manually writing epoll(7)</code></a> loops, and hand-roll state machines. At some point you may start thinking about creating abstractions to make these state machines compose better with each other, at which point you've basically arrived at futures again. Natively, in order to achieve: "I want to make this code run concurrently", threads are the easiest, most convenient abstraction available in non-async Rust.</p> </div> // thread-based concurrent computation </span>let</span> x = thread::spawn(\|\| </span>1 </span>+ </span>1</span>); </span>let</span> y = thread::spawn(\|\| </span>2 </span>+ </span>2</span>); </span>let </span>(x, y) = (x.</span>join</span>(), y.</span>join</span>()); </span>// wait for both threads to return </span> </span>// async-based concurrent computation </span>let</span> x = async { </span>1 </span>+ </span>1 </span>}; </span>let</span> y = async { </span>2 </span>+ </span>2 </span>}; </span>let </span>(x, y) = (x, y).await; </span>// resolve both futures concurrently </span></code></pre> This may seem like a pretty silly example: computation is synchronous and so both do the same thing, but the non-async variant has the overhead of needing to spawn actual threads. And it doesn't stop there: because the second example doesn't need threads, the compiler's inliner can kick in, and may be able to optimize it to the following 10</a></sup>:</p> // compiler-optimized async-based concurrent computation </span>let </span>(x, y) = (</span>2</span>, </span>4</span>); </span></code></pre> In contrast, the best optimization the compiler can likely perform for the thread-based variant is:</p> // thread-based concurrent computation </span>let</span> x = thread::spawn(\|\| </span>2</span>); </span>let</span> y = thread::spawn(\|\| </span>4</span>); </span>let </span>(x, y) = (x.</span>join</span>(), y.</span>join</span>()); </span>// wait for both threads to return </span></code></pre> ^10</sup>Note that this exact example doesn't yet work in the optimizer, but that I don't believe there is any strong reason why it couldn't either. It's all local reasoning, with no cross-thread synchronization needed. The closest example I have for optimizations of this kind is this example of a hand-rolled version of block_on</code></a> which compiles down to absolutely nothing. This cheats a little bit by removing not using an atomic-based Arc</code>, so I'm not sure how realistic it is. But it's def something to aspire to, and I'm optimistic that as async Rust sees more usage, we'll see more async-specific optimizations as well.</p> </div> Separating concurrency from parallelism allows for more optimizations of computations. async</code> in Rust is basically a fancy way of authoring a state machine, and nested async/.await</code> calls allow types to be compiled down into singular state machines. Sometimes we may want to separate the state machines</a> though, but that's the sort of control which async Rust provides us with, which is harder to achieve using non-async Rust.</p> Ecosystem</h2> Before we close out, we should point out one last reason people may choose async Rust: the size of the ecosystem. Without keyword generics</a> it can be a lot of work for library authors to publish and maintain libraries which work both in async and non-async Rust. Often it's easiest to just publish either an async or non-async library, and not account for the other use case. A lot of the network-related libraries on crates.io use async Rust though, which means that libraries building on top of this will also use async Rust. And in turn people looking to build websites without rewriting everything from scratch will often have a larger ecosystem to choose from when using async Rust.</p> Network effects are real and need to be acknowledged in this context. Not everyone wanting to build a website will be thinking in terms of language features, but may instead just be looking at the options they have in terms of ecosystem. And that's a perfectly valid reason to use async Rust for too.</p> Conclusion</h2> Sometimes conversations pop up about async Rust with suggestions such as: "What if we disallowed cancellation in its entirety?" 11</a></sup> Seeing this always confuses me, because it seems it carries a fundamental misunderstanding of what async Rust provides and why it should be used. If the focus is solely on performance, features such as cancellation or .await</code> annotations may seem like a plain nuisance.</p> ^{11</sup> The goal of this is often to have linear-futures, or "futures which are guaranteed to complete". The way by which cancellation would be disallowed is only the "stop polling" kind. You should still be able to pass channels around to cancel things, at least that's the theory. Though I believe there may also be implications for concurrency, which would make this really tough.</p> </div> But if the focus is more on the features async Rust enables, things like cancellation and timeouts quickly rise from nuisance to key reasons to adopt async Rust. Async Rust grants us the ability to control execution in a way which just isn't possible in non-async Rust. And frankly, isn't even possible in many other programming languages featuring async/.await</code> either. The fact that an async fn</code> compiles down to a lazy state machine instead of an eager managed task is a crucial distinction. And it means that we can author concurrency primitives entirely in library code, rather than needing to build it into the compiler or runtime.</p> In async Rust the features it enables build on each other. Here's a brief summary of how they relate:</p>} Yosh's Hierarchy </span> of Rust Async Capabilities </span> ┌───────────────────────────────┐ </span>3. │ Timeouts, Timers, Signals │ …which can then be composed into… </span> ├───────────────┬───────────────┤ </span>2. │ Cancellation │ Concurrency │ …which in turn enable… </span> ├───────────────┴───────────────┤ </span>1. │ Control over Execution │ The core futures enable… </span> └───────────────────────────────┘ </span></code></pre> We also briefly covered the performance aspect of async Rust. Generally speaking it can</em> be more performant than non-async Rust when you're doing async IO. But that will mostly be the case when the underlying system APIs are geared for that, which usually includes networking APIs, and more recently has also started including disk IO.</p> Thanks to Iryna Shestak</a> for proof reading this post and providing helpful feedback along the way.</em></p> Futures Concurrency IV: Join Ergonomics Mon, 19 Sep 2022 00:00:00 +0000 On Thursday this week Rust 1.64 will be released, and in it it will include a stabilized version of IntoFuture</code></a>. Much like IntoIterator</code> is used in the desugaring of for..in</code> loops, IntoFuture</code> will be used in the desugaring of .await</code>.</p> In this post I want to show some of the ergonomics improvements IntoFuture</code> might enable, inspired by Swift's recent improvements in async/await</code> ergonomics.</p> async let in swift</h2> Swift has recently added support for async let</code></a> 1</a></sup> in the language. This provides lightweight syntax to create Swift tasks, which before this proposal always had to be constructed explicitly within a task group</em>. In the evolution proposal they show an example like this:</p> ^1</sup>J. McCall, J. Groff, D. Gregor, and K. Malawski, “SE-0317: async let bindings,” Mar. 25, 2021</a>. (accessed Apr. 07, 2022).</p> </div> func </span>makeDinner() async -> Meal { </span> </span>// Create a task group to scope the lifetime of our three child tasks </span> </span>return try</span> await withThrowingTaskGroup(of: CookingTask.</span>self</span>) { group </span>in </span> </span>// spawn three cooking tasks and execute them in parallel: </span> group.async { </span> CookingTask.veggies(</span>try</span> await chopVegetables()) </span> } </span> group.async { </span> CookingTask.meat(await marinateMeat()) </span> } </span> group.async { </span> CookingTask.oven(await preheatOven(temperature: </span>350</span>)) </span> } </span> </span>// ... a whole lot more code after this </span>} </span></code></pre> That's quite a bit of code. And async let</code> exists to simplify that. Instead of explicitly creating the task group and manually creating tasks within it, using async let</code> the right scope is inferred for us 2</a></sup>, and instead the concurrency is expressed at the await</code> point later on:</p> ^{2</sup> I may be wrong on the details here. It's been a sec since I last read the Swift post, and I only glanced over the details today for this post. It shouldn't matter too much tho for the purpose of this post since we're mostly focusing on the end-user experience.</p> </div>}func </span>makeDinner() async throws -> Meal { </span> async </span>let</span> veggies = chopVegetables() </span> async </span>let</span> meat = marinateMeat() </span> async </span>let</span> oven = preheatOven(temperature: </span>350</span>) </span> </span> </span>let</span> dish = Dish(ingredients: await [</span>try</span> veggies, meat]) </span>// notice the concurrent await here </span> </span>return try</span> await oven.cook(dish, duration: .hours(</span>3</span>)) </span>} </span></code></pre> This feels very elegant in comparison, and definitely feels like a step up for Swift. I think the flow of this is so good in fact, that we may want to adopt something similar in Rust.</p> Join in Rust</h2> In Rust we already have a notion of a "concurrent await" through the join</code> operation. If you do async Rust you've likely seen it before in macro-form as join!</code></a> or the join_all</code></a> free-function:</p> // example of the `join!` macro: </span>let</span> a = future::ready(</span>1</span>u8</span>); </span>let</span> b = future::ready("</span>hello</span>"); </span>let</span> c = future::ready(</span>3</span>u16</span>); </span>assert_eq!(join!(a, b, c).await, (</span>1</span>, "</span>hello</span>", </span>3</span>)); </span> </span>// example of the `join_all` free-function: </span>let</span> futs = vec![</span>ready</span>(</span>1</span>u8</span>), </span>foo</span>(</span>2</span>u8</span>), </span>foo</span>(</span>3</span>u8</span>)]; </span>assert_eq!(</span>join_all</span>(futs).await, [</span>1</span>, </span>2</span>, </span>3</span>]); </span></code></pre> In my Futures Concurrency II</a> 3</a></sup> post and library</a> I show that instead of using a combination of macros and free functions we can instead use traits to extend container types with the necessary concurrency operations. This results in a smoother experience overall, since arrays, tuples, and vectors all operate in the same way:</p> ^3</sup>Y. Wuyts, “Futures Concurrency II,” Sep. 02, 2021</a>. (accessed Apr. 11, 2022).</p> </div> // alternative to the `join!` macro: </span>let</span> a = future::ready(</span>1</span>u8</span>); </span>let</span> b = future::ready("</span>hello</span>"); </span>let</span> c = future::ready(</span>3</span>u16</span>); </span>assert_eq!((a, b, c).</span>join</span>().await, (</span>1</span>, "</span>hello</span>", </span>3</span>)); </span> </span>// alternative to the `join_all` free-function: </span>let</span> a = future::ready(</span>1</span>); </span>let</span> b = future::ready(</span>2</span>); </span>let</span> c = future::ready(</span>3</span>); </span>assert_eq!(vec![a, b, c].</span>join</span>().await, vec![</span>1</span>, </span>2</span>, </span>3</span>]); </span></code></pre> This works fine, but it's not quite as good as what swift has using async let</code>. Let's take a look at how we can improve that.</p> Join Ergonomics</h2> Awaiting a single future is done by calling .await</code>, but awaiting multiple futures requires calling an intermediate method. To get join</code> semantics you have to call the join</code> method, rather than just being able to await a tuple of futures.</p> This is where IntoFuture</code> can help: we can use it to implement a conversion to a future directly on tuples, arrays, and vectors - and make it so if they contain futures you can just call .await</code> on them directly to await all futures concurrently. With that in place the above example could be rewritten like this:</p> let</span> a = future::ready(</span>1</span>u8</span>); </span>let</span> b = future::ready("</span>hello</span>"); </span>let</span> c = future::ready(</span>3</span>u16</span>); </span>assert_eq!((a, b, c).await, (</span>1</span>, "</span>hello</span>", </span>3</span>)); </span>// no more `.join()` needed </span> </span>let</span> a = future::ready(</span>1</span>); </span>let</span> b = future::ready(</span>2</span>); </span>let</span> c = future::ready(</span>3</span>); </span>assert_eq!(vec![a, b, c].await, vec![</span>1</span>, </span>2</span>, </span>3</span>]); </span>// no more `.join()` needed </span></code></pre> While being substantially different under the hood, the surface-level experience of this is actually quite similar to Swift's async let</code>. We just don't operate by default on tasks owned by a runtime, but futures which are lazy and compile down to in-line state machines. But this could be made to trivially work with multi-threaded Rust tasks too, if we follow the spawn approach laid out by tasky</a> (blog post</a> 4</a></sup>) 5</a></sup>:</p> ^4</sup>Y. Wuyts, “Postfix Spawn,” Mar. 04, 2022</a>. (accessed Sep. 18, 2022).</p> </div> ^{5</sup> I should explain this model in more detail at some point. The core of it is that we treat parallelism as a resource, and concurrency as a way of scheduling work. We mark futures which are "parallelizable" as such, and then pass them to the existing concurrency operators just like any other future. That creates a unified approach to concurrency for both parallel and non-parallel workloads.</p> </div>}let</span> a = future::ready(</span>1</span>u8</span>).</span>spawn</span>(); </span>let</span> b = future::ready("</span>hello</span>").</span>spawn</span>(); </span>let</span> c = future::ready(</span>3</span>u16</span>).</span>spawn</span>(); </span>assert_eq!((a, b, c).await, (</span>1</span>, "</span>hello</span>", </span>3</span>)); </span>// parallelized `await` </span> </span>let</span> a = future::ready(</span>1</span>).</span>spawn</span>(); </span>let</span> b = future::ready(</span>2</span>).</span>spawn</span>(); </span>let</span> c = future::ready(</span>3</span>).</span>spawn</span>(); </span>assert_eq!(vec![a, b, c].await, vec![</span>1</span>, </span>2</span>, </span>3</span>]); </span>// parallelized `await` </span></code></pre> This, to me, feels like a pretty good outcome for concurrent and parallel awaiting of fixed-sized sets of futures. Awaiting sets of futures which change over time is a different problem, and will likely require something akin to Swift's TaskSet</code> to function. But that's not most concurrent workloads, and I think we can take a page out of Swift's book on this.</p> Other Operations</h2> In a past post</a> 6</a></sup> I've shown the following table of concurrency operations for futures:</p> ^6</sup>Y. Wuyts, “Futures Concurrency III: select,” Feb. 09, 2022</a>. (accessed Apr. 06, 2022).</p> </div> </th> Wait for all outputs</strong></th> Wait for first output</strong></th></tr></thead> Continue on error</strong></td> Future::join</code></td> Future::try_race</code></td></tr> Return early on error</strong></td> Future::try_join</code></td> Future::race</code></td></tr> </tbody></table> join</code> is only one of 4 different concurrency operations. When you have different futures you may in fact want to do different things with it. But not all operations are equal: when we have two or more futures, it's usually because we're interested in observing all of their output. join</code> gives us exactly that. We'll get to try_</code> variants in the next section, but for now take for the following example:</p> let</span> a = future::ready(</span>1</span>); </span>let</span> b = future::ready(</span>2</span>); </span>let</span> a = a.await; </span>let</span> b = b.await; </span></code></pre> This first awaits a</code>, and then it awaits b</code>. Because the two are unrelated, it's often more efficient to await them concurrently</em> rather than sequentially</em>. With (async) IntoIterator</code> we're handed tools required to operate over values sequentially. But with IntoFuture</code> we're handed tools to operate over values concurrently</em> ^{7</a></sup>:</p>} ^{7</sup> IntoIterator</code> is not implemented for tuples, only for arrays and vectors. To make this example work we'll just use arrays. Perhaps we can one day use tuples too like this, but that may require having variadic tuples and maybe even structurally-typed enums for it to work.</p> </div>}// `IntoIterator` enables sequential operation </span>let</span> a = </span>1</span>; </span>let</span> b = </span>2</span>; </span>for</span> num in [a, b] { .. } </span>// sequential iteration </span> </span>// `IntoFuture` enables concurrent operation </span>let</span> a = future::ready(</span>1</span>); </span>let</span> b = future::ready(</span>2</span>); </span>let </span>[a, b] = [a, b].await; </span>// concurrent await </span></code></pre> The design of IntoFuture</code> intentionally mirrors the design of IntoIterator</code>. For any type T</code> we can only have a single IntoIterator</code> implementation. This leaves us with a few options:</p> don't implement IntoFuture</code> for containers</li> implement join</code> semantics for containers</li> implement race</code> semantics for containers</li> </ol> This post is argues that 2.</code> is a chance to provide better ergonomics than 1.</code>. But what about 3.</code>? Could we meaningfully have race</code> semantics as the default? race</code> takes N futures and yields one result. This would look like this:</p> let</span> a = future::ready(</span>1</span>u8</span>); </span>let</span> b = future::ready(</span>2</span>u8</span>); </span>let</span> c: </span>u8 </span>= [a, b].await; </span>// `race`-semantics </span></code></pre> I've said it before, but this seems like a minority use case. We'll almost always want a AND b</code>. Not a XOR b</code>. At least: we may still want to drop the values later on, but it's usually not decided based on which completed first. join</code> also feels like it extends the existing behavior of await</code> for individual futures to sets of futures. race</code> exposes different semantics than a plain .await</code> does, and so if awaiting tuples of futures was different, the resulting experience would feel inconsistent.</p> It seems good to have join</code> be the default semantics exposed through IntoFuture</code>, but then have operations such as race</code> and merge</code> be explicit forms of concurrency you can instead opt into by calling the right method.</p> Fallibility</h2> There is one other issue with the proposal: we have no way to expose fallbility of try_join</code> semantics. try_join</code> allows short-circuiting a join</code> operation if any of the fallible futures returns a Result::Error</code>. Right now we don't have a way to paramaterize the await</code> over fallibility, and since we only have a single IntoFuture</code> implementation we're stuck with just join</code>, which is not ideal. Perhaps using keyword generics</a> 8</a></sup> we can find a way out of this. After all: iterators face many of the same challenges today, since we can't just call for ?..in</code> in loops. With keyword generics we might be able to choose try_join</code> semantics for constructs such as:</p> ^8</sup>Y. Wuyts, “Announcing the Keyword Generics Initiative”</a>. (accessed Sep. 18, 2022).</p> </div> let</span> a = future::ready(Ok(</span>1</span>)); </span>let</span> b = future::ready(Ok(</span>2</span>)); </span>let </span>(a, b) = (a, b).await?; </span>// try_join await semantics </span></code></pre> This would infer that the want the fallible veriant of IntoFuture</code> since we call ?</code> on it after, and we're in a fallible context. But it's early on and hard to say precisely how this would work. It seems encouraging though that wanting fallible semantics for a construct is a shared problem across most of the language, so there is a desire to solve it in a unified way.</p> Conclusion</h2> In this post I've shown how Swift's async let</code> allows concurrent awaiting, how we can implement it in Rust through IntoFuture</code>, and what some of the challenges with it might be. I think Swift has done a great job at showing how convenient it is to be able to perform lightweight concurrent awaits, and I think we can learn from their approach and make it our own.</p> Before starting this post I didn't think too deeply about the similarities between IntoIterator</code> and IntoFuture</code>. But the more I look at them, the closer they seem. The fact that we have IntoIterator</code> for most every container might give us a hint for how we may want to think about container types. I'm not saying we should go out and implement IntoFuture</code> for HashMap</code> - but for vec, array, and tuples it definitely seems to make sense.</p> It seems that in terms of ordering we aren't too far out from being able to write an RFC for all of this either. I'm feeling pretty confident we've got the basic subset done for fixed-length async concurrency operators. And with IntoFuture</code> exposing join</code> semantics for arrays, tuples, and vectors, we can make the main method of concurrency easy to use as well. We'll have to see exactly what we want to do about tuples wrt the design - since they can't yet be variadic. And we'll want to wait on async traits to land first. But after 3 years of writing and experimentation on this topic, we don't seem far out 9</a></sup> from finally being able to propose something concrete for inclusion in the stdlib. And I think that's pretty exciting!</p> ^9</sup>The join!</code> macro</a> was added to the stdlib a while back through a regular libs PR. But any stabilization of it should be blocked on fleshing out a complete concurrency model for async Rust</a>.</p> </div> State Machines II: an enum-based design Tue, 23 Aug 2022 00:00:00 +0000 In the past I've written about state machines</a> in Rust. And more recently I've also written about anonymous enums</a> 1</a></sup> in Rust. I've been wondering what would happen if the two interacted? I think the result could actually be quite nice, and worth daydreaming about. In this post we'll be discussing state machines, the language features which could make them easier to use, and ways in which we could push the ergonomics further.</p> ^{1</sup> "anonymous enums" are also known as "sum types". There might be subtle differences, but I'll be using them interchangeably. For the context of this post though, we assume that taking two members of the same enum and putting them in a sum type results in a sub-set of the original enum rather than creating a new enum with two members. For the sake of brevity in this post, just assume that works as intended.</p> </div>}The case for state machines</h2> A lot of programming is about going from one state to another. We can do that using conditionals statements, but as the state changes those conditions can be tricky to reason about and in turn refactor. Instead it's often more convenient to use state machines</em>: singular objects which represent all possible states and their transitions. And in compiled languages specifically to use compiler-checked state machines</em> 2</a></sup>.</p> ^2</sup>See my JavaScript nanostate</a> library for an example of a state machine library which is not checked by a compiler - instead performing all checks at runtime.</p> </div> Say we wanted to implement a basic green-orange-red traffic light that can also flash. We can imagine the rules for transitions between states could be the following:</p> Green can turn to orange</li> Orange can turn to red</li> Red can turn to green</li> Red can turn to flashing red 3</a></sup></li> Flashing red can turn to red</li> </ol> ^{3</sup> If you've never seen a traffic light blink red before: this is common for traffic lights in the Netherlands. It can indicate that the traffic control systems are malfunctioning. Or it can be used to indicate the traffic lights have been disabled, which can be done late at night when there's hardly any traffic to be managed anyway.</p> </div> All states and their transitions can be represented by the following graph:</p> </p> For the remainder of this post we'll be implementing the rules of the graph above. Our goal is to implement a compiler-checked state machine which can inform us during compilation whether all our state transitions are valid, or if we've accidentally written an invalid transition which we need to fix.</p>}Representing State Machines in Rust In The Future</h2> To read about how we can write compiler-checked state machines in Rust today</em> you can read my previous post on state machines</a>. In this post I instead want to focus on ✨ the future ✨. How could we plausibly represent this in Rust by extending the language. In the "future directions" section of the last post we wrote a state machine without the blinky lights. Using enum variants types</a> (not yet a thing) combined with arbitrary self-types</a> (also not yet a thing), we could write the following:</p> //! A traffic light without a flashing state. </span>//! Novel language features: enum-variant-types, arbitrary-self-types </span> </span>enum </span>State { </span> Green, </span> Orange, </span> Red, </span>} </span> </span>impl </span>State { </span> </span>pub fn </span>new</span>() -> </span>Self::</span>Green { </span> </span>Self</span>::Green </span> } </span> </span> </span>pub fn </span>next</span>(</span>self</span>: </span>Self::</span>Green) -> </span>Self::</span>Orange { </span> </span>Self</span>::Orange </span> } </span> </span> </span>pub fn </span>next</span>(</span>self</span>: </span>Self::</span>Orange) -> </span>Self::</span>Red { </span> </span>Self</span>::Red </span> } </span> </span> </span>pub fn </span>next</span>(</span>self</span>: </span>Self::</span>Red) -> </span>Self::</span>Green { </span> </span>Self</span>::Green </span> } </span>} </span> </span>fn </span>main</span>() { </span> </span>let mut</span> state = State::new(); </span>// green </span> state = state.</span>next</span>(); </span>// orange </span> state = state.</span>next</span>(); </span>// red </span> state = state.</span>next</span>(); </span>// green </span>} </span></code></pre> What I didn't realize at the time I wrote my last post is that this in effect acts like a form of overloading</em>. Functionally the code we've written isn't different than writing a single next</code> function which internally updates the state. But instead we expose the state externally</em>. The following example will be written in valid Rust today</em>, exposing the exact same functionality as our last example — but with the state only observable internally</em>:</p> //! A traffic light without a flashing state. </span>//! Novel language features: none </span> </span>enum </span>State { </span> Green, </span> Orange, </span> Red, </span>} </span> </span>impl </span>State { </span> </span>pub fn </span>new</span>() -> </span>Self::</span>Green { </span> </span>Self</span>::Green </span> } </span> </span> </span>pub fn </span>next</span>(</span>self</span>) -> </span>Self </span>{ </span> </span>match </span>self </span>{ </span> </span>Self</span>::Green => </span>Self</span>::Orange, </span> </span>Self</span>::Orange => </span>Self</span>::Red, </span> </span>Self</span>::Red => </span>Self</span>::Green, </span> } </span> } </span>} </span> </span>fn </span>main</span>() { </span> </span>let mut</span> state = State::new(); </span>// green </span> state = state.</span>next</span>(); </span>// orange </span> state = state.</span>next</span>(); </span>// red </span> state = state.</span>next</span>(); </span>// green </span>} </span></code></pre> Let's continue with the internally observable state examples for a bit. Our graph shows we need to support a flashing</code> state, so we need to add that. We need some way to trigger going into a flashing state, so we'll add a new method: flash</code>. Not all states can transition into the flashing state, so we need to make sure we can only go into it from "red" - which means we need to panic if we attempt to transition into it from any other state:</p> //! A traffic light with a flashing state. </span>//! Novel language features: none </span> </span>enum </span>State { </span> Green, </span> Orange, </span> Red, </span> Flashing, </span>} </span> </span>impl </span>State { </span> </span>pub fn </span>new</span>() -> </span>Self::</span>Green { </span> </span>Self</span>::Green </span> } </span> </span> </span>pub fn </span>next</span>(</span>self</span>) -> </span>Self </span>{ </span> </span>match </span>self </span>{ </span> </span>Self</span>::Green => </span>Self</span>::Orange, </span> </span>Self</span>::Orange => </span>Self</span>::Red, </span> </span>Self</span>::Red => </span>Self</span>::Green, </span> </span>Self</span>::Flashing => </span>Self</span>::Red, </span> } </span> } </span> </span> </span>pub fn </span>flash</span>(</span>self</span>) -> </span>Self </span>{ </span> </span>match </span>self </span>{ </span> State::Red => </span>Self</span>::Flashing </span> _ => panic!("</span>only red can turn into flashing red</span>"), </span> } </span> } </span>} </span> </span>fn </span>main</span>() { </span> </span>let mut</span> state = State::new(); </span>// green </span> state = state.</span>next</span>(); </span>// orange </span> state = state.</span>next</span>(); </span>// red </span> state = state.</span>flash</span>(); </span>// flashing </span> state = state.</span>next</span>(); </span>// red </span>} </span></code></pre> This is not ideal: we now have added a runtime check to a state machine which was checked entirely during compilation before. The reason a runtime check is needed is because by adding a method which isn't available on all types, the internal</em> state becomes observable externally</em>. And the Rust language lacks the tools to conveniently represent this 4</a></sup>.</p> ^{4</sup> Yes, I know we can refactor the whole thing and use traits and generics to hack together something which gives us similar semantics. I've shown how to do that in previous posts. The point of this post is not "can we represent state machines using Rust's type system"</em> (yes we can), but "which language features are we lacking to make it easy to represent state machines in Rust's type system"</em>.</p> </div> So let's try and rewrite this again using enum variant types and arbitrary self-types. Here the caller knows exactly what state the enum is in, and we can more precisely represent the states and state transitions:</p>}//! A traffic light with a flashing state. </span>//! Novel language features: enum-variant-types, arbitrary-self-types </span> </span>enum </span>State { </span> Green, </span> Orange, </span> Red, </span> Flashing, </span>} </span> </span>impl </span>State { </span> </span>pub fn </span>new</span>() -> </span>Self::</span>Green { </span> </span>Self</span>::Green </span> } </span> </span> </span>pub fn </span>next</span>(</span>self</span>: </span>Self::</span>Green) -> </span>Self::</span>Orange { </span> </span>Self</span>::Orange </span> } </span> </span> </span>pub fn </span>next</span>(</span>self</span>: </span>Self::</span>Orange) -> </span>Self::</span>Red { </span> </span>Self</span>::Red </span> } </span> </span> </span>pub fn </span>next</span>(</span>self</span>: </span>Self::</span>Red) -> </span>Self::</span>Green { </span> </span>Self</span>::Green </span> } </span> </span> </span>// new </span> </span>pub fn </span>next</span>(</span>self</span>: </span>Self::</span>Flashing) -> </span>Self::</span>Red { </span> </span>Self</span>::Red </span> } </span> </span> </span>// new </span> </span>pub fn </span>flash</span>(</span>self</span>: </span>Self::</span>Red) -> </span>Self::</span>Flashing { </span> </span>Self</span>::Flashing </span> } </span>} </span> </span>fn </span>main</span>() { </span> </span>let mut</span> state = State::new(); </span>// green </span> state = state.</span>next</span>(); </span>// orange </span> state = state.</span>next</span>(); </span>// red </span> state = state.</span>flash</span>(); </span>// flashing </span> state = state.</span>next</span>(); </span>// red </span>} </span></code></pre> This would allow us to correctly model and validate the entire state graph at compile-time, without needing to resort to any runtime checks. Yay us!</p> Ergonomics: Overview</h2> Hopefully folks can see the benefit of being able to model state transitions solely through the type system. Obviously there are other ways of authoring compiler-checked state machines today - but that's not the point of this post. What we're trying to think about is how we can do so in a way which feels simple</em> and dare I even say, intuitive</em> for people to use.</p> I think what we've shown so far is pretty good. But it has a few shortcomings, namely:</p> It's pretty verbose to individually write each transition</li> What happens when we have more than one valid return state from a function?</li> </ol> Okay, so bear with me for a sec here. In the graph each line goes from a source</em> to a destination</em>. In the code the sources are represented by the self-types, and the destinations are represented by the return types. This means that any node in the graph with more than arrow pointing from or to it could potentially benefit from refactoring. In the graph above there are instances of this:</p> orange</code> and flashing</code> point to red</code></li> red</code> points to both green</code> and flashing</code></li> </ol> For any type system enthusiasts reading this post: yes we're going to be talking about representing powerset enums natively using Rust's typesystem here 5</a></sup>.</p> ^5</sup>For everyone who isn't a type system enthusiast and doesn't know about "powerset enums", it's basically just "subsets of enums". Instead of declaring that a method can return all</em> variants in an enum, using powersets we can declare that a method can only return a specific subset. Check out the powerset_enum crate</a> to read more about it. Or alternatively you can keep reading, because that's basically what we'll be implementing.</p> </div> Ergonomics: Merging Self-Types</h2> Now let's start with the first one: "orange" and "flashing" both point to "red". So far we've been talking about arbitrary self-types as a feature. But there are ideas floating around about expanding this for structs, called: view types</a>. What this allows you to do is inform the borrow checker we don't actually need access to the entire struct - we only need access to parts of it. So we can hold multiple mutable borrows on the same struct as long as the fields don't overlap.</p> Now what if we had the equivalent of these struct "views" but for enums. Instead of "viewing" subsets of fields, we'd be able to "view" subsets of enums. Now let's make up a syntax to show what we mean. In patterns we already have Foo \| Bar</code> to represent multiple patterns, so let's reuse that here:</p> // before </span>impl </span>State { </span> </span>pub fn </span>next</span>(</span>self</span>: </span>Self::</span>Orange) -> </span>Self::</span>Red { </span> </span>Self</span>::Red </span> } </span> </span> </span>pub fn </span>next</span>(</span>self</span>: </span>Self::</span>Flashing) -> </span>Self::</span>Red { </span> </span>Self</span>::Red </span> } </span>} </span></code></pre> // after </span>impl </span>State { </span> </span>pub fn </span>next</span>(</span>self</span>: </span>Self::</span>Orange \| </span>Self::</span>Flashing) -> </span>Self::</span>Red { </span> </span>Self</span>::Red </span> } </span>} </span></code></pre> I know many of you will likely have opinions about the syntax, but let's not focus on that for a minute. Instead I want to focus your attention attention on the semantics at play here: it would bring the same pattern semantics as used in match</code> statements to methods, which I believe should feel a lot more natural?</p> Riffing time. There are some real questions about what syntax view types should use</a>. Niko's post suggests something like &{golden_tickets} self</code>. But what if instead we chose to reuse the pattern syntax for the self</code> types 6</a></sup> - which could yield us something like &Self { golden_tickets, .. }</code>. This would reuse the pattern-destructuring syntax. Which is similar to what we're doing here: reusing the pattern enum matching syntax. It feels like there might be something to it?</p> ^6</sup>My colleague eric</a> suggested exploring this direction a while back. And I actually quite like it! (Direct any credit to Eric, but blame to me please. I've definitely run with this way past what he likely intended.)</p> </div> The compiler knows that the possible members of self</code> in the example ("orange", "flashing") are only a subset of all members in State</code>. What if the compiler actually used this information in the function body too? That would mean matching on self</code> would only need to account for the possible</em> cases rather than all</em> cases:</p> match </span>self </span>{ </span> </span>Self</span>::Orange => {} </span> </span>Self</span>::Flashing => {} </span> </span>// no other cases need matching! </span>} </span></code></pre> This isn't particularly relevant yet for the simple traffic light example we're dealing with here, but definitely becomes a lot more relevant when working with real-world code. No longer needing to panic!</code> for branches you know can't be reached anyway would make code both simpler and faster, which is a pretty big deal!</p> Ergonomics: Merging Return Types</h2> The second case is: "red can point to both green and flashing"</em>. This one will be a bit harder to reason about because instead of taking multiple different parameters we're trying to return</em> multiple parameters. This means the compiler can no longer statically</em> know which variant we have, and instead we have to perform runtime matching to figure that out. Right now the code we have looks like this:</p> // before </span>impl </span>State { </span> </span>pub fn </span>next</span>(</span>self</span>: </span>Self::</span>Red) -> </span>Self::</span>Green { </span> </span>Self</span>::Green </span> } </span> </span> </span>pub fn </span>flash</span>(</span>self</span>: </span>Self::</span>Red) -> </span>Self::</span>Flashing { </span> </span>Self</span>::Flashing </span> } </span>} </span></code></pre> We have two different methods which take Self::Red</code> and return a type. For argument's sake say we wanted a single method with took some data to determine whether we should go to green</code> or flashing</code> 7</a></sup>. This could then look something like this:</p> ^{7</sup> "For argument's sake" means I couldn't come up with a good example. So this is def a cop-out.</p> </div>}// after </span>impl </span>State { </span> </span>pub fn </span>next</span>(</span>self</span>: </span>Self::</span>Red, </span>data</span>: Data) -> </span>Self::</span>Green \| </span>Self</span>::Flashing { </span> </span>if </span>validate_data</span>(data) { </span> </span>Self</span>::Green </span> } </span>else </span>{ </span> </span>Self</span>::Flashing </span> } </span> } </span>} </span></code></pre> Again, the syntax is not really the point. The idea here is though that just like in the earlier example the returned value would be known to be a subset of Self</code>: it would still register as the State</code> enum, but the only two accessible members are Green</code> and Flashing</code> 8</a></sup>.</p> ^{8</sup> Maybe a better way to signal this would be to use something like -> Self { Green \| Flashing }</code> as syntax? Idk,, I'm focusing more on semantics than syntax right now.</p> </div> On the usage side things would change too: we could no longer statically know which state we're in. All we know is that the state is either green or flashing. This leads to some very real questions: should we be able to then call next</code> again to get to orange \| red</code>?</p>}fn </span>main</span>() { </span> </span>let mut</span> state = State::new(); </span>// green </span> state = state.</span>next</span>(); </span>// orange </span> state = state.</span>next</span>(); </span>// red </span> state = state.</span>next</span>(data); </span>// green \| flashing - ok I guess? </span> state = state.</span>next</span>(); </span>// orange \| red - uhhh,, </span> state = state.</span>next</span>(?data); </span>// green \| red \| flashing - x_x </span>} </span></code></pre> Once we're in this state, we have different next</code> methods. How do we decide how we should call them? Even if we can answer this question, the semantics of this seem pretty complex and hard to teach. So instead I propose we apply some basic rules here to simplify usage:</p> Once a variable is assigned to a single enum-member-as-value, it can only be re-assigned to other single values and vice-versa.9</a></sup></li> Methods with the same name on the same enum need to have the same "base" (type from which a view is derived) signature.</li> </ol> ^{9</sup> Maybe this rule could be relaxed over time? I'm not sure; it seems very graph-problem-y, and I don't know if there are any great solutions to this.</p> </div> The first rule would disallow assigning green \| flashing</code> to state</code>. Instead we'd need to match</code> to know which variant we have, and go from there:</p>}//! Never assign an enum view-type to `state`. This requires matching on the data </span>//! until we've extraced a single member. Assume we wait for conditions to change </span>//! between calls to `state.next`. </span> </span>fn </span>main</span>() { </span> </span>let mut</span> state = State::new(); </span>// green </span> state = state.</span>next</span>(); </span>// orange </span> state = state.</span>next</span>(); </span>// red </span> state = </span>handle_flash</span>(state.</span>next</span>(&data), &data); </span>// green </span>} </span> </span>/// Keep looping until we hit a green state. </span>fn </span>handle_flash</span>(</span>mut </span>state</span>: </span>State::</span>Green \| </span>State::</span>Flashing, </span>data</span>: &Data) -> </span>State::</span>Green { </span> </span>match</span> state { </span> State::Green => State::Green, </span> State::Flashing => { </span> state = state.</span>next</span>(); </span>// red </span> </span>to_green</span>(state.</span>next</span>(&data), &data) </span>// green </span> } </span> } </span> </span>} </span></code></pre> There are a bunch of assumptions baked in here - not in the least that an anonymous enum of State::Green \| State::Flashing</code> can be passed to an enum view-type matching the same members. But we're not doing details in this post. We're doing vibes. Yeehaw.</p> Okay, so for the second rule we're proposing: methods with the same name on the same enum should share the same base signature</em>. This means that methods which return a view</em> of some enum or struct are fine - because they're the same "base" type. But having different methods with different signatures like we've done here with next(data)</code> would be a no-no. This would allow us to keep thinking about methods which take or return enum views as a form of specialization.</p> That last point is perhaps not strictly required, but it has the benefit of making the enum feel more consistent</em>. And it could lead allow for specialization-like behavior where if you implement the same method for all enum members, it then becomes unconditionally available on the entire enum - since it can be called regardless of which state you're in, even if it yields different behavior. But on a more intuitive level: it feels pretty weird that self.next()</code> sometimes takes an argument and sometimes doesn't. It kind of feels like parameter overloading like it's done in other languages, and Rust kind of steers clear of that. So it feels better to just enforce that methods with the same name on the same enum need to take the same arguments - even if we can wiggle around with view-types and sub-types a little.</p> Tying It Together</h2> Phew, that's been a ride. The last section has been a lot about what not</em> to do. So let's show an example of how things might look when we put them all together. Let's start with what we had before, but slightly adapted to make the flash</code> case conditional:</p> //! A traffic light with a flashing state. </span>//! Novel language features: enum-variant-types, arbitrary-self-types </span> </span>enum </span>State { </span> Green, </span> Orange, </span> Red, </span> Flashing, </span>} </span> </span>impl </span>State { </span> </span>pub fn </span>new</span>() -> </span>Self::</span>Green { </span> </span>Self</span>::Green </span> } </span> </span> </span>pub fn </span>next</span>(</span>self</span>: </span>Self::</span>Green) -> </span>Self::</span>Orange { </span> </span>Self</span>::Orange </span> } </span> </span> </span>pub fn </span>next</span>(</span>self</span>: </span>Self::</span>Orange) -> </span>Self::</span>Red { </span> </span>Self</span>::Red </span> } </span> </span> </span>pub fn </span>next</span>(</span>self</span>: </span>Self::</span>Red) -> </span>Self::</span>Green { </span> </span>Self</span>::Green </span> } </span> </span> </span>// new </span> </span>pub fn </span>next</span>(</span>self</span>: </span>Self::</span>Flashing) -> </span>Self::</span>Red { </span> </span>Self</span>::Red </span> } </span> </span> </span>// new </span> </span>pub fn </span>flash</span>(</span>self</span>: </span>Self::</span>Red) -> </span>Self::</span>Flashing { </span> </span>Self</span>::Flashing </span> } </span>} </span> </span>fn </span>main</span>() { </span> </span>let mut</span> state = State::new(); </span>// green </span> state = state.</span>next</span>(); </span>// orange </span> state = state.</span>next</span>(); </span>// red </span> </span>if </span>validate_data</span>(data) { </span> state = </span>self</span>.</span>next</span>(); </span>// green </span> </span>// handle green </span> } </span>else </span>{ </span> state = </span>self</span>.</span>flash</span>(); </span>// flashing </span> </span>// handle flashing </span> } </span> } </span>} </span></code></pre> Now if we translate that to our enum views variant, we're able to inline the conditional statement into the enum. And together with the arbitrary self enum pattern syntax, the overall code is a lot more trimmed. And especially the usage should feel a bit more natural:</p> //! A traffic light with a flashing state. </span>//! Novel language features: enum-variant-types, arbitrary-self-types, </span>//! enum-view-types, enum-sum-types </span> </span>enum </span>State { </span> Green, </span> Orange, </span> Red, </span> Flashing, </span>} </span> </span>impl </span>State { </span> </span>pub fn </span>new</span>() -> </span>Self::</span>Green { </span> </span>Self</span>::Green </span> } </span> </span> </span>pub fn </span>next</span>(</span>self</span>: </span>Self::</span>Green) -> </span>Self::</span>Orange { </span> </span>Self</span>::Orange </span> } </span> </span> </span>pub fn </span>next</span>(</span>self</span>: </span>Self::</span>Orange \| </span>Self::</span>Flashing) -> </span>Self::</span>Red { </span> </span>Self</span>::Red </span> } </span> </span> </span>// New: Inlines the logic of the previous example to decide whether to go </span> </span>// from "red" to "green" or "flashing". </span> </span>pub fn </span>try_next</span>(</span>self</span>: </span>Self::</span>Red, </span>data</span>: Data) -> </span>Self::</span>Green \| </span>Self</span>::Flashing { </span> </span>if </span>validate_data</span>(data) { </span> </span>Self</span>::Green </span> } </span>else </span>{ </span> </span>Self</span>::Flashing </span> } </span> } </span>} </span> </span>fn </span>main</span>() { </span> </span>let mut</span> state = State::new(); </span>// green </span> state = state.</span>next</span>(); </span>// orange </span> state = state.</span>next</span>(); </span>// red </span> state.</span>try_next</span>(data) { </span> State::Green => {}, </span>// handle green </span> State::Flashing => {}, </span>// handle flashing </span> }; </span>} </span></code></pre> Conclusion</h2> In this post we've looked at how using "enum variant types", "enum sum types", and "enum view types" could allow us to make state machines at the type level in Rust easier to author and more ergonomic. The example we've used here has been a variation of a basic traffic light. But despite that we expect that it should be possible to adopt the patterns and features shown here to state machines in general.</p> Challenges</strong></p> It seems that taking enums as self-types should be relatively straight-forward. But returning anonymous enums is more complicated. I came up with some rules to help circumvent some of the sharp edges, but I feel those probably could use some more thought. Maybe folks reading this have some insights on it?</p> Making Enums More Useful</strong></p> Also as my colleague Arlie</a> pointed out during a recent chat: having enums-as-types would be really beneficial for other reasons too. Take for example the syn</code> crate 10</a></sup>: the WherePredicate</code></a> has a member Type</code> which contains a struct called PredicateType</code></a>. The reason why we have a separate struct instead of just inlining the data in the enum member, is because members can't be used as types. So if we want to author functions which operate on the data in WherePredicate::Type</code>, we need</em> to create a separate struct 11</a></sup>. This makes record-enums less useful than their struct counterparts, and complicates the API surface area of crates.</p> ^{10</sup> I picked this as an example, so if there are actually different instances where PredicateType</code> is used - that's on me. I'm not an expert on syn</code> so details may be wrong. But I hope the example is clear enough that at least the point comes across: because this does happen in a lot of places.</p> </div>}^{11</sup> Ideally we should only create need to create separate structs if we want to reuse types between enums. That's not the case today.</p> </div> Structural Typing and Errors</strong></p>}While we're on the topic of anonymous enums in general: another important use case for it would be for error handling. Anonymous enums provide us with structural typing for enums 12</a></sup>, which allows for convenient composition of error types. Say we have a ParseError</code></a> and io::Error</code></a>, a function should just be able to specify -> Result<T, ParseError \| io::Error></code>. But more importantly: we should be able to specify which error variants can be returned. This is a bit harder with io::Error</code> specifically because of how it's constructed. But would work great when using errors returned by thiserror</code></a>. I've been meaning to write about error handling ever since I posted my error handling survey</a> three years ago. Maybe I'll do that later this year.</p> ^{12</sup> You can think of "structural typing" as "matching on the shape" instead of "matching on the type". Matching on a tuple with (first, second)</code> is structural: we don't care what types are in the tuple, we mostly care about the shape. It still needs to type-check of course, but combined with type inference it can give us a lot of flexibility. For example TypeScript's type system is structural, and in turn provides a lot of flexibility.</p> </div> View Types</strong></p>}Also, while writing this post I realized that being able to use enum variants as self-types is kind of analogous to view types for structs. Both carve out subsets of their base type, but for different purposes. Which makes sense because they're actually different. But as a general trend the implications seem fun: by more precisely articulating which sub-types we use, the compiler can allow more code to function. I think the reason for that is that function boundaries act as strict guards for what's allowed, and as a rule are also pretty coarse-grained. By making the more expressive, the compiler becomes less strict, and in turn more code can be allowed. It really does feel like view types and the like might represent a generational leap akin to NLL</a> in terms of general language ergonomics. I think it's interesting to think about what else could be loosened within the framework of Rust's existing semantics.</p> Goals of this post</strong></p> To re-iterate: this post is intentionally incomplete. I've knowingly glossed over details, and left pieces out. The goal of the post was to show what the outcomes</em> we could have if we combined some features folks are thinking about anyway. Together with a personal goal to not spend months writing about this, because I actually should be working on diffferent things. Rust's roadmap is always a balance of team prioritization and personal interests; so showing what can be achieved by combining several features might lower the bar to make the case these features would be useful, or perhaps even get people interested in working on this.</p> Wrapping up</strong></p> Anyway, that's probably enough discussion for now. This post was a massive procrastination from writing another post on "halt points", which I've finished drafting and am currently editing. This was is mostly thoughts which have been stewing in the back of my mind after talking with colleages at work about state machines, type systems, and more accurately modeling programs. Thanks for reading!</p> Announcing the Keyword Generics Initiative Wed, 27 Jul 2022 00:00:00 +0000 This post is a mirror of the post published to the Inside Rust blog</a>. If you want to read an earlier iteration of the ideas described in this post, check out the post on Async Overloading</a>, written about a year earlier.</p> </blockquote> We (Oli</a>, Niko</a>, and Yosh</a>) are excited to announce the start of the Keyword Generics Initiative</a>, a new initiative 1</a></sup> under the purview of the language team. We're officially just a few weeks old now, and in this post we want to briefly share why we've started this initiative, and share some insight on what we're about.</p> ^{1</sup> Rust governance terminology can sometimes get confusing. An "initiative" in Rust parlance is different from a "working group" or "team". Initiatives are intentionally limited: they exist to explore, design, and implement specific pieces of work - and once that work comes to a close, the initiative will wind back down. This is different from, say, the lang team - which essentially carries a 'static</code> lifetime - and whose work does not have a clearly defined end.</p> </div>}A missing kind of generic</h2> One of Rust's defining features is the ability to write functions which are generic</em> over their input types. That allows us to write a function once, leaving it up to the compiler to generate the right implementations for us.</p> Rust allows you to be generic over types - it does not allow you to be generic over other things that are usually specified by keywords. For example, whether a function is async, whether a function can fail or not, whether a function is const or not, etc.</p> The post "What color is your function"</a> 2</a></sup> describes what happens when a language introduces async functions, but with no way to be generic over them:</p> I will take async-await over bare callbacks or futures any day of the week. But we’re lying to ourselves if we think all of our troubles are gone. As soon as you start trying to write higher-order functions, or reuse code, you’re right back to realizing color is still there, bleeding all over your codebase.</p> </blockquote> This isn't just limited to async though, it applies to all modifier keywords - including ones we may define in the future. So we're looking to fill that gap by exploring something we call "keyword generics" 3</a></sup>: the ability to be generic over keywords such as const</code> and async</code>.</p> ^{2</sup> R. Nystrom, “What Color is Your Function?,” Feb. 01, 2015. https://journal.stuffwithstuff.com/2015/02/01/what-color-is-your-function/ (accessed Apr. 06, 2022).</p> </div>}^{3</sup> The longer, more specific name would be: "keyword modifier generics". We've tried calling it that, but it's a bit of a mouthful. So we're just sticking with "keyword generics" for now, even if the name for this feature may end up being called something more specific in the reference and documentation.</p> </div> To give you a quick taste of what we're working on, this is roughly how we imagine you may be able to write a function which is generic over "asyncness" in the future:</p>} Please note that this syntax is entirely made up, just so we can use something in examples. Before we can work on syntax we need to finalize the semantics, and we're not there yet. This means the syntax will likely be subject to change over time.</p> </blockquote> async<A> </span>trait </span>Read { </span> async<A> </span>fn </span>read</span>(&</span>mut </span>self</span>, </span>buf</span>: &</span>mut</span> [</span>u8</span>]) -> Result<</span>usize</span>>; </span> async<A> </span>fn </span>read_to_string</span>(&</span>mut </span>self</span>, </span>buf</span>: &</span>mut</span> String) -> Result<</span>usize</span>> { ... } </span>} </span> </span>/// Read from a reader into a string. </span>async<A> </span>fn </span>read_to_string</span>(</span>reader</span>: &</span>mut</span> impl Read * A) -> std::io::Result<String> { </span> </span>let mut</span> string = String::new(); </span> reader.</span>read_to_string</span>(&</span>mut</span> string).await?; </span> string </span>} </span></code></pre> This function introduces a "keyword generic" parameter into the function of A</code>. You can think of this as a flag which indicates whether the function is being compiled in an async context or not. The parameter A</code> is forwarded to the impl Read</code>, making that conditional on "asyncness" as well.</p> In the function body you can see a .await</code> call. Because the .await</code> keyword marks cancellation sites</a> we unfortunately can't just infer them 4</a></sup>. Instead we require them to be written for when the code is compiled in async mode, but are essentially reduced to a no-op in non-async mode.</p> ^{4</sup> No really, we can't just infer them - and it may not be as simple as omitting all .await</code> calls either. The Async WG is working through the full spectrum of cancellation sites, async drop, and more. But for now we're working under the assumption that .await</code> will remain relevant going forward. And even in the off chance that it isn't, fallibility has similar requirements at the call site as async does.</p> </div> We still have lots of details left to figure out, but we hope this at least shows the general feel</em> of what we're going for.</p>}A peek at the past: horrors before const</h2> Rust didn't always have const fn</code> as part of the language. A long long long long long time ago (2018) we had to write a regular function for runtime computations and associated const of generic type logic for compile-time computations. As an example, to add the number 1</code> to a constant provided to you, you had to write (playground</a>):</p> trait </span>Const<T> { </span> </span>const </span>VAL</span>: T; </span>} </span> </span>/// `42` as a "const" (type) generic: </span>struct </span>FourtyTwo; </span>impl </span>Const<</span>i32</span>> </span>for </span>FourtyTwo { </span> </span>const </span>VAL</span>: </span>i32 </span>= </span>42</span>; </span>} </span> </span>/// `C` -> `C + 1` operation: </span>struct </span>AddOne<C: Const<</span>i32</span>>>(C); </span>impl</span><C: Const<</span>i32</span>>> Const<</span>i32</span>> </span>for </span>AddOne<C> { </span> </span>const </span>VAL</span>: </span>i32 </span>= C::</span>VAL </span>+ </span>1</span>; </span>} </span> </span>AddOne::<FourtyTwo>::</span>VAL </span></code></pre> Today this is as easy as writing a const fn</code>:</p> const fn </span>add_one</span>(</span>i</span>: </span>i32</span>) -> </span>i32 </span>{ </span> i + </span>1 </span>} </span> </span>add_one</span>(</span>42</span>) </span></code></pre> The interesting part here is that you can also just call this function in runtime code, which means the implementation is shared between both const</code> (CTFE5</a></sup>) and non-const</code> (runtime) contexts.</p> ^{5</sup> CTFE stands for "Compile Time Function Execution": const</code> functions can be evaluated during compilation, which is implemented using a Rust interpreter (miri).</p> </div>}Memories of the present: async today</h2> People write duplicate code for async/non-async with the only difference being the async</code> keyword. A good example of that code today is async-std</code></a>, which duplicates and translates a large part of the stdlib's API surface to be async 6</a></sup>. And because the Async WG has made it an explicit goal to bring async Rust up to par with non-async Rust</a>, the issue of code duplication is particularly relevant for the Async WG as well. Nobody on the Async WG seems particularly keen on proposing we add a second instance of just about every API currently in the stdlib.</p> ^{6</sup> Some limitations in async-std</code> apply: async Rust is missing async Drop</code>, async traits, and async closures. So not all APIs could be duplicated. Also async-std</code> explicitly didn't reimplement any of the collection APIs to be async-aware, which means users are subject to the "sandwich problem". The purpose of async-std</code> was to be a proving ground to test whether creating an async mirror of the stdlib would be possible: and it's proven that it is, as far as was possible with missing language features.</p> </div>}We're in a similar situation with async</code> today as const</code> was prior to 2018. Duplicating entire interfaces and wrapping them in block_on</code> calls is the approach taken by e.g. the mongodb</code> [async</a>, non-async</a>], postgres</code> [async</a>, non-async</a>], and reqwest</code> [async</a>, non-async</a>] crates:</p> // Async functionality like this would typically be exposed from a crate "foo": </span>async </span>fn </span>bar</span>() -> Bar { </span> </span>// async implementation goes here </span>} </span></code></pre> // And a sync counterpart would typically be exposed from a crate </span>// named "blocking_foo" or a submodule on the original crate as </span>// "foo::blocking". This wraps the async code in a `block_on` call: </span>fn </span>bar</span>() -> Bar { </span> futures::executor::block_on(foo::bar()) </span>} </span></code></pre> This situation is not ideal. Instead of using the host's synchronous syscalls, we're now going through an async runtime to get the same results - something which is often not zero-cost. But more importantly, it's rather hard to keep both a sync and async API version of the same crate in, err, sync with each other. Without automation it's really easy for the two APIs to get out of sync, leading to mismatched functionality.</p> The ecosystem has come up with some solutions to this, perhaps most notably the proc-macro based maybe-async</code> crate</a>. Instead of writing two separate copies of foo</code>, it generates a sync and async variant for you:</p> #[</span>maybe_async</span>] </span>async </span>fn </span>foo</span>() -> Bar { ... } </span></code></pre> While being useful, the macro has clear limitations with respect to diagnostics and ergonomics. That's absolutely not an issue with the crate, but an inherent property of the problem it's trying to solve. Implementing a way to be generic over the async</code> keyword is something which will affect the language in many ways, and a type system + compiler will be better equipped to handle it than proc macros reasonably can.</p> A taste of trouble: the sandwich problem</h2> A pervasive issue in existing Rust is the sandwich</em> problem. It occurs when a type passed into an operation wants to perform control flow not supported by the type it's passed into. Thus creating a sandwich</em> 7</a></sup> The classic example is a map</code> operation:</p> ^{7</sup> Not to be confused with the higher-order sandwich dilemma</em> which is when you look at the sandwich problem and attempt to determine whether the sandwich is two slices of bread with a topping in between, or two toppings with a slice of bread in between. Imo the operation part of the problem feels more bready</em>, but that would make for a weird-looking sandwich. Ergo: sandwich dilemma. (yes, you can ignore all of this.)</p> </div> enum </span>Option<T> { </span> Some(T), </span> None, </span>} </span> </span>impl</span><T> Option<T> { </span> </span>fn </span>map</span><J>(</span>self</span>, </span>f</span>: impl FnOnce(</span>T</span>) -> J) -> Option<J> { ... } </span>} </span> </span>my_option.</span>map</span>(\|</span>x</span>\| x.await) </span></code></pre> This will produce a compiler error: the closure f</code> is not an async context, so .await</code> cannot be used within it. And we can't just convert the closure to be async</code> either, since fn map</code> doesn't know how to call async functions. In order to solve this issue, we could provide a new async_map</code> method which does</em> provide an async closure. But we may want to repeat those for more effects, and that would result in a combinatorial explosion of effects. Take for example "can fail" and "can be async":</p> </th> not async</th> async</th></tr></thead> infallible</strong></td> fn map</code></td> fn async_map</code></td></tr> fallible</strong></td> fn try_map</code></td> fn async_try_map</code></td></tr> </tbody></table> That's a lot of API surface for just a single method, and that problem multiplies across the entire API surface in the stdlib</strong>. We expect that once we start applying "keyword generics" to traits, we will be able to solve the sandwich problem. The type f</code> would be marked generic over a set of effects, and the compiler would choose the right variant during compilation.</p> Affecting all effects</h2> Both const</code> and async</code> share a very similar issue, and we expect that other "effects" will face the same issue. "fallibility" is particularly on our mind here, but it isn't the only effect. In order for the language to feel consistent we need consistent solutions.</p> FAQ</h2> Q: Is there an RFC available to read?</h3> Rust initiatives are intended for exploration</em>. The announcement of the Keyword Generics Initiative marks the start</em> of the exploration process. Part of exploring is not knowing what the outcomes will be. Right now we're in the "pre-RFC" phase of design. What we hope we'll achieve is to enumerate the full problem space, design space, find a balanced solution, and eventually summarize that in the form of an RFC. Then after the RFC is accepted: implement it on nightly, work out the kinks, and eventually move to stabilize. But we may at any point during this process conclude that this initiative is actually infeasible and start ramping down.</p> But while we can't make any assurances</em> about the outcome of the initiative, what we can share is that we're pretty optimistic about the initiative overall. We wouldn't be investing the time we are on this if we didn't think we'd be actually be able to see it through to completion.</p> Q: Will this make the language more complicated?</h3> The goal of keyword generics is not to minimize the complexity of the Rust programming language, but to minimize the complexity of programming in Rust.</em> These two might sound similar, but they're not. Our reasoning here is that by adding</em> a feature, we will actually be able to significantly reduce the surface area of the stdlib, crates.io libraries, and user code - leading to a more streamlined user experience.</p> Choosing between sync or async code is a fundamental choice which needs to be made. This is complexity which cannot be avoided, and which needs to exist somewhere. Currently in Rust that complexity is thrust entirely on users of Rust, making them responsible for choosing whether their code should support async Rust or not. But other languages have made diferent choices. For example Go doesn't distinguish between "sync" and "async" code, and has a runtime which is able to remove that distinction.</p> In today's Rust application authors choose whether their application will be sync or async, and even after the introduction of keyword generics we don't really expect that to change. All generics eventually need to have their types known, and keyword generics are no different. What we're targeting is the choice made by library</em> authors whether their library supports is sync or async. With keyword generics library authors will be able to support both with the help of the compiler, and leave it up to application authors to decide how they want to compile their code.</p> Q: Are you building an effect system?</h3> The short answer is: kind of, but not really. "Effect systems" or "algebraic effect systems" generally have a lot of surface area. A common example of what effects allow you to do is implement your own try/catch</code> mechanism. What we're working on is intentionally limited to built-in keywords only, and wouldn't allow you to implement anything like that at all.</p> What we do share with effect systems is that we're integrating modifier keywords more directly into the type system. Modifier keywords like async</code> are often referred to as "effects", so being able to be conditional over them in composable ways effectively gives us an "effect algebra". But that's very different from "generalized effect systems" in other languages.</p> Q: Are you looking at other keywords beyond async</code> and const</code>?</h3> For a while we were referring to the initiative as "modifier generics" or "modifier keyword generics", but it never really stuck. We're only really interested in keywords which modify how types work. Right now this is const</code> and async</code> because that's what's most relevant for the const-generics WG and async WG. But we're designing the feature with other keywords in mind as well.</p> The one most at the top of our mind is a future keyword for fallibility. There is talk about introducing try fn() {}</code> or fn () -> throws</code> syntax. This could make it so methods such as Iterator::filter</code> would be able to use ?</code> to break out of the closure and short-circuit iteration.</p> Our main motiviation for this feature is that without it, it's easy for Rust to start to feel disjointed</em>. We sometimes joke that Rust is actually 3-5 languages in a trenchcoat. Between const rust, fallible rust, async rust, unsafe rust - it can be easy for common APIs to only be available in one variant of the language, but not in others. We hope that with this feature we can start to systematically close those gaps, leading to a more consistent Rust experience for all</em> Rust users.</p> Q: What will the backwards compatibility story be like?</h3> Rust has pretty strict backwards-compatibility guarantees, and any feature we implement needs to adhere to this. Luckily we have some wiggle room because of the edition mechanism, but our goal is to shoot for maximal backwards compat. We have some ideas of how we're going to make this work though, and we're cautiously optimistic we might actually be able to pull this off.</p> But to be frank: this is by far one of the hardest aspects of this feature, and we're lucky that we're not designing any of this just by ourselves, but have the support of the language team as well.</p> Q: Aren't implementations sometimes fundamentally different?</h3> Const Rust can't make any assumptions about the host it runs on, so it can't do anything platform-specific. This includes using more efficient instructions of system calls which are only available in one platform but not another. In order to work around this, the const_eval_select</code></a> intrinsic in the standard library enables const</code> code to detect whether it's executing during CTFE or runtime, and execute different code based on that.</p> For async we expect to be able to add a similar intrinsic, allowing library authors to detect whether code is being compiled as sync or async, and do something different based on that. This includes: using internal concurrency, or switching to a different set of system calls. We're not sure whether an intrinsic is the right choice for this though; we may want to provide a more ergonomic API for this instead. But because keyword generics is being designed as a consistent feature, we expect that whatever we end up going with can be used consistently by all</em> modifier keywords.</p> Conclusion</h2> In this post we've introduced the new keyword generics initiatve, explained why it exists, and shown a brief example of what it might look like in the future.</p> The initiative is active on the Rust Zulip under t-lang/keyword-generics</code></a> - if this seems interesting to you, please pop by!</p> Thanks to everyone who's helped review this post, but in particular: fee1-dead</a>, Daniel Henry-Mantilla</a>, and Ryan Levick</a></em></p> Keywords I: Unsafe Syntax Sat, 09 Jul 2022 00:00:00 +0000 Last week Oli</a> and I were thinking through some examples involving modifiers and impl</code> blocks. Namely: would it be possible or even useful to declare something like this:</p> // all methods in this block are `const`. </span>const impl </span>MyStruct { </span> </span>fn </span>foo</span>(&</span>mut </span>self</span>) {} </span> </span>fn </span>bar</span>(&</span>mut </span>self</span>) {} </span>} </span></code></pre> We thought this looked alright. But something we realized is that if we allowed this for const</code>, we'd likely want to allow this for all modifier keywords too, including unsafe</code>. And that unfortunately causes some issues. So this post is a brief look at what those issues are, what plans exist to improve it, and how we might even be able to do things better?</p> Note: The purpose of this post is mostly to share thoughts. I don't speak for Oli here, and I definitely don't speak for any Rust team. Don't take think of this as a serious proposal, but instead think of it as notes which may come in useful at a later time.</em></p> Two meanings of unsafe</h2> unsafe</code></a> in Rust is actually two keywords in a trenchcoat. Depending on where you use it, it'll mean something different:</p> On definition: unsafe fn</code> / unsafe trait</code></li> On usage: unsafe {}</code> / unsafe impl</code></li> </ul> Each provides the following contract:</p> On definition: there are additional invariants which the compiler cannot check, so the programmer must check those by hand instead.</li> On usage: I'm manually checking invariants here which the compiler cannot check for me.</li> </ul> Together these form a pair - not unlike async/.await</code>. With the slight difference that defining an unsafe</code> block in a function body does not come with the additional restriction that the function itself must also be marked as unsafe</code>:</p> </span>// using `.await` requires you wrap it in an `async` context, which propagates </span> </span>// to the next caller: </span>async </span>fn </span>foo</span>() { </span> </span>bar</span>().await; </span>} </span> </span>// however using `unsafe` doesn't require you wrap it in an `unsafe` context; </span>// `fn foo` doesn't also need to be `unsafe fn foo` here: </span>fn </span>foo</span>() { </span> </span>unsafe </span>{ </span>bar</span>() }; </span>} </span></code></pre> And this makes sense: pretty much all of Rust builds on some foundation of unsafe</code>. So if unsafe</code> was transitive much the same way async/.await</code> is, then virtually every function would need to be marked unsafe</code> - which would defeat the point of using unsafe</code> as a sign post for where we're manually checking invariants.</p> Bringing it back to traits</h2> So okay, that's all well and good. But I mentioned we got onto this because we were exploring the idea of allow keyword modifiers on entire blocks. Which is a useful thing to have if, idk, you're in the process of marking the majority of the stdlib as const</code> 1</a></sup>. If you're able to declare things in a more general way it makes things, uh, flow a bit more.</p> ^{1</sup> In practice very few APIs cannot ever be called from const fn</code> functions. Those are: all APIs which need to have knowledge of the host system (OS, filesystem, network devices, etc.) or have access to statics. I think that's like, less than 10% of the total API surface in the stdlib; meaning the vast majority may eventually be constified. Not even to being about adding other types of effects (cough</em> async cough</em>) to the stdlib as well.</p> </div> But here's where it gets odd. Take the following code:</p>}struct </span>MyStruct; </span> </span>unsafe impl </span>Send </span>for </span>MyStruct {} </span></code></pre> This says: "we promise the implementation of Send</code> here is safe". But if we adapt our const impl</code> example from earlier to be unsafe</code> impl instead we get this:</p> // The intended meaning here is: </span>// "all methods in this block are `unsafe` to call" </span>unsafe impl </span>Foo { </span> </span>fn </span>foo</span>(&</span>mut </span>self</span>) {} </span> </span>fn </span>bar</span>(&</span>mut </span>self</span>) {} </span>} </span></code></pre> This has the exact opposite meaning of our earlier unsafe impl</code>! Doing this means we'd have a semantic difference between inherent impls 2</a></sup> and trait impls. Which is a suuuuuper subtle distinction, and one which is almost guaranteed to trip people up!</p> ^{2</sup> This is jargon for: "A regular impl block on a type". So not the trait impl block.</p> </div>}Do we have options?</h2> There's some things we might be able to do here. First option:</p> Do nothing!</p> </blockquote> Yes, we could definitely choose to not care about this and move on. But that's an intellectual dead end, and I don't like those. I'm choosing to care about this instead. At least enough to write about it and think about it and think things through a bit more.</p> Color within the lines!</p> </blockquote> So what if we choose to restrict ourselves to making only additive changes here. No breaking changes allowed. Well, we could yield that unsafe impl</code> has the meaning it has, and we can't change that. So maybe we could for example change where we put the keyword to have different meanings. What if we could make it something like this instead:</p> struct </span>MyStruct; </span> </span>// `unsafe impl` - this impl has been checked </span>unsafe impl </span>Send </span>for </span>MyStruct {} </span> </span>// `impl unsafe` - the methods here need to be checked when used </span>impl </span>unsafe MyStruct { </span> </span>fn </span>foo</span>(&</span>mut </span>self</span>) {} </span> </span>fn </span>bar</span>(&</span>mut </span>self</span>) {} </span>} </span></code></pre> We could make it so unsafe impl</code> means: "this impl has been checked". And impl unsafe</code> means: "this implementation needs to be checked. That's still subtle, but at least it would no longer be identical. Adapting it to const</code> would then be:</p> // `const impl` would be disallowed; that's for `unsafe` only. </span>impl </span>const MyStruct { </span> </span>fn </span>foo</span>(&</span>mut </span>self</span>) {} </span> </span>fn </span>bar</span>(&</span>mut </span>self</span>) {} </span>} </span></code></pre> This is workable, but imo not ideal. When I say things out loud I definitely say it's a "const impl" or "async impl". But we'd be writing it the other way around.</p> Okay, so what if we start coloring outside the lines?</h2> What if we actually took a step back and asked ourselves: "if we could start from scratch, what would we change about unsafe</code>?" Well, for starters, I think our memory model</a> is looking pretty good! So we wouldn't want to bother too much there, and focus more on the syntax.</p> Today having an unsafe fn</code> implies the function body to be an unsafe {}</code> block. RFC 2585 (2020)</a> was merged to break that assumption; presumably over the course of a few editions (though afaict no decisions have been made on the exact process). But that would move us one step closer to differentiating between the two meanings of unsafe</code>.</p> But what if we deprecated unsafe</code> entirely instead? RFC 117 (2014)</a> suggested we do exactly this, albeit pre-1.0. But given the semantics</em> of unsafe</code> wouldn't change, but only the syntax would, this is the type of change I believe we could run over an edition bound.</p> Personally I like checked</code> / unchecked</code> to differentate between the two types of unsafe</code> 3</a></sup>. It's common terminology</a> in the stdlib today already, and it would solve the problem of "what did you mean by unsafe</code>" entirely. Our code examples could then be written like this:</p> ^3</sup>I like "checked" more than I like "safe". When I hear safe</em> to me it sounds like we've promised there are no bugs. And to quote Dijkstra: Testing shows the presence, not the absence of bugs</a>. At best</em> we can guarantee we've checked for bugs. And its up to others to decide whether they deem that safe enough.</p> </div> struct </span>MyStruct; </span> </span>// this implementation has been checked </span>checked </span>impl </span>Send </span>for </span>MyStruct {} </span> </span>// these methods need to be checked </span>unchecked </span>impl </span>MyStruct { </span> </span>fn </span>foo</span>(&</span>mut </span>self</span>) {} </span> </span>fn </span>bar</span>(&</span>mut </span>self</span>) {} </span>} </span></code></pre> See! No more ambiguity. And I think that'd be a huge step up from the status quo! To spell out the other uses as well, here's all existing uses of unsafe</code> translated to use "check" terminology:</p> // `unsafe fn` </span>unchecked </span>fn </span>foo</span>() {} </span> </span>// `unsafe trait` </span>unchecked </span>trait </span>MyTrait {} </span> </span>// `unsafe impl` </span>checked </span>impl </span>Send </span>for </span>MyStruct {} </span> </span>// `unsafe {}` </span>fn </span>foo</span>() { </span> checked { </span>bar</span>() }; </span>} </span></code></pre> But owo there's more!</h2> What? There's more?</p> Yes! Because we now have a keyword pair, we can take a page out of the book of other keyword pairs. Ergonomics matter in all parts of a language, but especially so in places where the stakes are high. So what if we allowed</p> In RFC 2585 Niko</a> showed that many unsafe</code> functions today are one-liners, and making it so unsafe fn</code> does not imply unsafe {}</code> would yield the following result:</p> unsafe fn </span>foo</span>(...) -> ... { </span>unsafe </span>{ </span> </span>// Code goes here </span>} } </span></code></pre> Now, what if... we did away with unsafe {}</code> blocks (or at least supplemented them) and added a postfix operator operator? What if we could do this instead!:</p> unchecked </span>fn </span>foo</span>(...) -> ... { </span> </span>bar</span>().checked </span>} </span></code></pre> We can probably debate exact words and syntax all day long. But the idea doesn't seem too bad, does it? Now granted, perhaps there are solid reasons to keep blocks as well. But it's common advice that unsafe</code> blocks should be scoped as tightly as possible. In today's rust that often means we're writing things like:</p> unsafe fn </span>foo</span>(...) -> ... { </span> </span>unsafe </span>{ </span>bar</span>() }.</span>baz</span>() </span>} </span></code></pre> And I think in comparison having a postfix notation would look a lot better:</p> unchecked </span>fn </span>foo</span>(...) -> ... { </span> </span>bar</span>().checked.</span>baz</span>() </span>} </span></code></pre> Going even further?</h2> The core principle of Rust is that it can wrap unchecked internals into a package which exposes checked semantics. This allows us to clearly scope which parts of the program need to deal with manually checking invariants, and allows the rest of the program to not have to care about any of that. Instead the compiler helps us.</p> But not all checks are equal. I trust the stdlib devs to have carefully poured over every single line of unsafe</code> and done their best to ensure it's correct. At worst there might be a bug. But when I'm downloading code from third parties, well, we can be exposed to far more than bugs</a> 4</a></sup>.</p> ^{4</sup> I believe this is the first supply chain attack we've become aware of. It does something bad during compilation. But a malicious crate might just as well choose to do something bad during runtime</em>. Supply chain attacks are real, and as Rust grows, so will the frequency of attacks.</p> </div> unsafe</code> is the building block that everything else is built on. Allowing all code we download to do anything at all is not great. I mean: it's great</em> insofar that we can wrap openssl</code>, libgit</code>, winapi</code>, and more. But we tend to vet those packages, and make sure they're from trusted sources. It's not great when that one random protocol parser I downloaded was actually turned out to be a piece of ransomware.</p>}Sunfish</a> wrote a thought experiment</a> about what it would take to create a sandbox backed by the Rust type system. And at least we'd need to have a knob to turn off giving unrestricted access to unsafe</code> to every function in every dependency we have.</p> pub fn </span>im_a_sweet_and_innocent_parser_uwu</span>(</span>reader</span>: impl Read) -> ParsedTree { </span> </span>haha_no_im_not_lemme_break_into_ur_unpatched_kernel_real_quick</span>(); </span> ParsedTree::new(reader) </span>} </span></code></pre> The reason why I'm raising it here is because in a way this goes in the opposite</em> direction of RFC 2585. Instead of decoupling unsafe fn</code> from unsafe {}</code> it almost creates a tighter relationship between the two - but through other means. In order to use unsafe {}</code> / checked {}</code> anywhere in a function, the function signature would need to reflect that.</p> We wouldn't want to make the unsafe</code> keyword transitive so this would need to behave differently than async/.await</code>. But in order to enable</em> unsafe</code> to be used, we'd want to share the capability</em> to do so via the function signature. The only types excluded from this mechanism would be the stdlib; since if you trust the compiler you should also trust the stdlib.</p> // `foo` needs maximum permissions ("ambient authority") to </span>// perform `unsafe`/`checked` operations: </span>fn </span>foo</span>() </span>with </span> ambient_authority </span>{ </span> </span>bar</span>().checked </span>} </span></code></pre> This would allow us to start flagging which functions depend on having the keys to the castle, and which do not. Obviously that's not the full story, capabilities and especially capability safety</em> are a pretty deep topic in and of themselves. But because they would affect the function signature of unsafe</code> types, it's worth bringing up here.</p> Conclusion</h2> Okay, so we kind of went hard on the changes we could plausibly want to make to unsafe</code> in the future. But I think that's fine! - We at least figured a plausible way out of the main drawback of RFC 2585 - though I guess a postfix unsafe</code> keyword could hold water even without the rename to checked/unchecked</code>.</p> But equally importantly this would actually open the path to allowing const impl {}</code>, async impl</code>, and possibly other effects in the future as well. And that's probably needed, because as we're looking to make our typesystem more powerful, we don't want people to have to actually type</em> more (get it) to make use of it.</p> This initially was a thread on twitter</a>, but I figured it might be worth giving it some permanence. I hope you enjoyed it; love you.</p> Async Cancellation II: Time and Signals Fri, 10 Jun 2022 00:00:00 +0000 For the past few years I've been working on the async-std</code> library, which provides an async implementation of the APIs exposed by std</code>. However, we also added several new APIs related to things unique to async Rust: concurrency, control over execution, and the interaction between the two.</p> These APIs were initially introduced in async-std</code> as "unstable", and have been the main focus of my work to design since. On this blog there are numerous posts related to for example: concurrency</a>, cancellation</a>, and parallelism</a>. Today I want to share a new experiment I've been working for time-based operations in async Rust. I've designed it as a stand-alone crate for now, but I intend to PR its addition to async-std</code> in the near future.</p> Timer basics</h2> Relevant APIs here are:</em></p> futures_time::task::Sleep</code></a></li> futures_time::stream::Interval</code></a></li> </ul> In order to do anything with time you need to have timers. There are two basic types of timers: a timer which fires once at a specific time, and a timer which fires repeatedly after each interval 1</a></sup>. In futures-time</code> you can create both these timers by concstructing these futures directly. But more interestingly: we allow people to construct these futures by convert an existing</em> time::Duration</code> or time::Instant</code> directly into a timer through the IntoFuture</code> and IntoStream</code> traits.</p> ^{1</sup> People would be right to point out that Interval</code> can be implemented in terms of Sleep</code>. But even though that's the case, APIs will have to choose between "do I fire once" or "do I fire repeatedly", and for that reason it's worth distinguishing between the two. Besides: Interval</code> is so common that we don't want folks to have to implement it over and over, so we just provide it as one of the two basic timer types.</p> </div> Because both these traits are also implemented for all T: Future</code> and T: Stream</code> respectively, time-based APIs can take timer: impl IntoFuture</code> or instant: impl IntoStream</code> and pass either Duration</code>/Instant</code> directly, or another future.</p>}use </span>futures_timer::prelude::; </span>use </span>futures_timer::{task, time::Duration}; </span> </span>async </span>fn </span>wait_until</span>(</span>timer</span>: impl IntoFuture) { </span> </span>let</span> timer = timer.</span>into_future</span>(); </span> timer.await; </span>} </span> </span>#[</span>async_std</span>::</span>main</span>] </span>async </span>fn </span>main</span>() { </span> </span>// Wait for a `Duration`. </span> </span>wait_until</span>(Duration::from_secs(</span>1</span>)); </span> </span> </span>// Wait for a concrete future. </span> </span>let</span> deadline = task::sleep(Duration::from_secs(</span>1</span>)); </span> </span>wait_until</span>(deadline); </span>} </span></code></pre> To simplify working with the orphan rules, we ended up defining our own variants of IntoFuture</code>, IntoStream</code>, Duration</code>, and Instant</code>. When we move this to async-std</code> we can probably simplify some of it - but we'll likely need to keep our own definitions of Duration</code> and Instant</code> just for this to keep working.</p> Cancelling execution at a distance</h2> Relevant APIs here are:</em></p> futures_time::future::FutureExt::timeout</code></a></li> futures_time::stream::StreamExt::timeout</code></a></li> </ul> As I showed in the last section, thanks to the implementation of the Into</code> traits functions can be agnostic over whether they take a Duration</code>/Instant</code> instance, or whether they take a Future</code>. For the last three years we've been trying to figure out how we could integrate the stop-token</code> crate</a> into async-std</code>.</p> For a deep-dive into how cancellation works in async Rust, you can read the last post in this series</a>. But the short version is: futures can be cancelled between .await</code> points just by dropping the future. A timeout is nothing more than a "race" between a future F</code> and a timer future. If F</code> completes first, you get an Ok(value)</code>. And if the timer completes first, you get an Err(TimeoutError)</code>. That's concepts composing nicely with each other!</p> We tried various APIs for cancellation, but none really felt good. So I figured I just needed to give it some time, and started looking at async temporal operations. When I saw RxJS's timeout</code></a> and timeoutWith</code></a> operators I realized we might actually be able to combine both functionality through a single API. And instead of using a dedicated StopSource</code></a> / StopToken</code></a> pair, we can instead reuse the async-channel</code> crate</a> to create a sender + receiver. This is what it looks like when used together:</p> use </span>futures_lite::prelude::; </span>use </span>futures_time::prelude::; </span>use </span>futures_time::channel; </span>use </span>futures_time::time::Duration; </span> </span>fn </span>main</span>() { </span> async_io::block_on(async { </span> </span>let </span>(send, </span>mut</span> recv) = channel::bounded::<()>(</span>1</span>); </span>// create a new send/receive pair </span> </span>let</span> value = async { "</span>meow</span>" } </span> .</span>delay</span>(Duration::from_millis(</span>100</span>)) </span> .</span>timeout</span>(recv.</span>next</span>()) </span>// time-out when the sender emits a message </span> .await; </span> </span> assert_eq!(value.</span>unwrap</span>(), "</span>meow</span>"); </span> }) </span>} </span></code></pre> This example will always return meow</em> because send</code> is never dropped. But in a concurrent example, a timeout could be triggered for all instances of recv</code> (which implements Clone</code>) by dropping send</code>. This is pretty nice because it combines several individually useful components to create "cancel at a distance semantics" which are usually reserved for dedicated APIs.</p> But there are still several rough edges in this API, which I think we may be able to address:</p> There are multiple imports from different crates required to get this to work. Though this will likely be smoothed out somewhat by exposing it from async-std</code>.</li> When constructing a channel you have to declare a mut recv</code> instead of recv</code>. This is because recv.next()</code> expects &mut self</code>. Implementing IntoFuture</code> for async_channel::Receiver</code> would remove that requirement, streamlining things somewhat.</li> channel::bounded</code> needs to know what type it is, so it takes a ::<()></code> turbo-fish. Creating zero-sized channels for messaging is useful on its own, so once we're able to </a> we may want the constructor to default to a zero-sized type.</li> </ol> If we migrate to async-std</code> and have all these changes applied, usage might end up looking somewhat like this:</p> use </span>async_std::prelude::; </span>use </span>async_std::channel; </span>use </span>async_std::time::Duration; </span> </span>#[</span>async_std</span>::</span>main</span>] </span>async </span>fn </span>main</span>() { </span> </span>let </span>(send, recv) = channel::bounded(</span>1</span>); </span> </span>let</span> value = async { "</span>meow</span>" } </span> .</span>delay</span>(Duration::from_millis(</span>100</span>)) </span> .</span>timeout</span>(recv) </span> .await; </span> </span> assert_eq!(value.</span>unwrap</span>(), "</span>meow</span>"); </span>} </span></code></pre> Delaying execution</h2> Relevant APIs here are:</em></p> futures_time::future::FutureExt::delay</code></a></li> futures_time::stream::StreamExt::delay</code></a></li> futures_time::future::FutureExt::park</code></a></li> futures_time::stream::StreamExt::park</code></a></li> </ul> Arbitrary cancellation of execution might be one of the most important features of async Rust, but it's not the only one. By gaining control over execution we also gain the ability to delay, pause, and resume execution as well. Here's an example of how to delay an execution by a set time limit, or until a signal is sent:</p> use </span>futures_time::prelude::; </span>use </span>futures_time::time::{Instant, Duration}; </span>use </span>std::future; </span> </span>fn </span>main</span>() { </span> async_io::block_on(async { </span> </span>let</span> value = future::ready("</span>meow</span>") </span> .</span>delay</span>(Duration::from_millis(</span>100</span>)) </span>// delay resolving the future by 100ms </span> .await; </span> assert_eq!(value, Some("</span>meow</span>")); </span> }); </span>} </span></code></pre> I just authored the park</code> methods for this post, but didn't get around to writing examples for them yet. park</code> works similar to std::thread::park</code></a>, but instead of only working on threads it works with any</em> future or stream. You call it by passing it a channel which can park or unpark the future, allowing for convenient pausing and resuming of execution.</p> In a previous post</a> I mentioned that I suspect the right analogy for "task" is "parallelizable future"</em> rather than "async thread"</em>, and showing that we can implement park</code> for all</em> futures rather than only on Task</code> is another indication that this new model may in fact be the right one.</p> Throttling execution</h2> Relevant APIs here are:</em></p> futures_time::stream::StreamExt::buffer</code></a></li> futures_time::stream::StreamExt::debounce</code></a></li> futures_time::stream::StreamExt::sample</code></a></li> futures_time::stream::StreamExt::throttle</code></a></li> </ul> Generally data loss is considered a bad thing. But sometimes… data loss can be a good thing? Take for example mouse presses: when something is clicked once, you want to handle that. But if something is pressed 15 times in .3 seconds, perhaps not so much. This is an example of "external input", and since we cannot apply backpressure to external conditions, the only method we have to reduce strain on our system is to actively drop load.</p> In the crate we provide some versatile, basic methods to shed load based on different circumstances. The debounce</code> and sample</code> methods get the last item seen during a particular period. The throttle</code> method will let through the first items received during a particular period. And finally buffer</code> groups data from a particular time range into a vector, allowing operators to operate on chunks of data rather than needing to decide on individual items.</p> use </span>futures_lite::prelude::; </span>use </span>futures_time::prelude::; </span>use </span>futures_time::time::{Instant, Duration}; </span>use </span>futures_time::stream; </span> </span>fn </span>main</span>() { </span> async_io::block_on(async { </span> </span>let mut</span> counter = </span>0</span>; </span> stream::interval(Duration::from_millis(</span>5</span>)) </span>// yield every 5ms </span> .</span>take</span>(</span>10</span>) </span>// stop after 10 iterations </span> .</span>buffer</span>(Duration::from_millis(</span>20</span>)) </span>// create 20ms windows </span> .</span>for_each</span>(\|</span>buf</span>\| counter += buf.</span>len</span>()) </span>// sum all items </span> .await; </span> </span> assert_eq!(counter, </span>10</span>); </span> }) </span>} </span></code></pre> Conclusion</h2> And that concludes the whirlwind-tour of the futures-time</code></a> crate. I'm pretty happy with the resulting APIs. For the longest time we were struggling to create APIs which could operate on {Duration, Instant, impl Future}</code>, and this usually resulted in needing to declare three different APIs for each piece of functionality. The insight that we could implement IntoFuture</code> for Instant</code> and Duration</code> closed that gap:</p> async </span>fn </span>run</span>(</span>mut </span>recv</span>: Receiver<()>) { </span> </span>// Notice how we're using the same API: </span> </span>let</span> db = </span>start_db</span>().</span>timeout</span>(Duration::from_secs(</span>10</span>)).await; </span>// Timeout based on a duration </span> </span>let</span> server = </span>start_server</span>().</span>timeout</span>(recv.</span>next</span>()).await; </span>// Timeout based on a signal </span>} </span></code></pre> One of the things I'm least sure about though is what it feels like to convert a Duration</code> to a Future</code> outside a trait bound. Admittedly this feels somewhat awkward to me:</p> use </span>futures_time::time::Duration; </span>use </span>futures_time::prelude::; </span> </span>async </span>fn </span>main</span>() { </span> Duration::from_secs(</span>2</span>).await; </span>// await a duration of 2 secs? </span>} </span></code></pre> I'm not sure. It might be the verb-noun thing, but it feels a bit weird. But remembering an API Sean McArthur proposed years ago, we might be able to streamline this a bit using conversion traits2</a></sup>:</p> ^2</sup>Note that IntoDuration</code> does not yet exist, but here's a playground link</a> to what that might look like.</p> </div> use </span>futures_time::time::Duration; </span>use </span>futures_time::prelude::; </span>use </span>into_duration::IntoDuration; </span> </span>async </span>fn </span>main</span>() { </span> </span>2.</span>secs</span>().await; </span>// wait for 2 secs. </span>} </span></code></pre> This reads a bit more fluent than what we might be used from most Rust APIs, but I don't think that's a bad thing. I think it just takes some getting used to. And it actually works quite nicely in other places too:</p> use </span>async_std::prelude::; </span>use </span>async_std::channel; </span>use </span>async_std::time::Duration; </span> </span>#[</span>async_std</span>::</span>main</span>] </span>async </span>fn </span>main</span>() { </span> </span>let</span> value = async { "</span>meow</span>" } </span> .</span>delay</span>(</span>100.</span>millis</span>()) </span>// <- using this API </span> .await; </span> </span> assert_eq!(value.</span>unwrap</span>(), "</span>meow</span>"); </span>} </span></code></pre> And thinking further ahead: if we get async IntoIterator</code> + async iteration syntax, having it implemented for Duration</code> could allow us to write this:</p> let</span> start = Instant::now(); </span>for</span> await now in </span>1.</span>secs</span>() { </span>// loop every 1 second </span> dbg!(now.</span>duration_since</span>(start)); </span>// print elapsed time </span>} </span></code></pre> Either way, there are some fun options available here. Implementing IntoFuture</code> / async IntoIterator</code> for Duration</code> is one of the things I'm still least sure about - but I wonder how much of it is just a matter of getting used to something novel that's now suddenly possible. I really don't know.</p> That said though; I'm really happy with the way the library has turned out. And I hope with some feedback and iteration it'll be good enough to include into async-std</code>, and eventually also help us inform whether and how we want to expose this functionality from the stdlib.</p> There are a bunch of time-based APIs we haven't covered. But we certainly have covered the fundamentals. And in particular I'm happy that we've finally found a promising API for remote cancellation / pausing / resumption of arbitrary futures. This is one of the key features async Rust provides, and actually providing APIs for them seems important.</p> Either way, that's what I have for now. If you want to check out the repo, you can find it here: yoshuawuyts/futures-time</code></a>. If you'd like to keep up with my thoughts in near-real-time you can follow me on Twitter</a>. And if you think the work I'm doing is useful and you'd like to send me a tip, you can sponsor me here</a>. Thanks!</p> Postfix Spawn Fri, 04 Mar 2022 00:00:00 +0000 Using a free spawn</code> function to create new tasks has been a staple</a> in async Rust for many years. When we introduced async-std</code></a> in 2019 we took this further by not only providing a spawn</code> function, but fully emulating the std::thread</code></a> family of APIs and returning. But just because something has been around for a while doesn't mean it's a good idea. And we know we have issues. In this post I want to cover some of those issues, and share an experiment I've been working on which may help with this.</p> Structured Concurrency</h2> In my previous post I covered the idea of structured concurrency</a>. Though it's worth expanding on this concept more in a separate post 1</a></sup>, here's the definition we'll be using for the sake of this post:</p> Child tasks should never outlive their parent tasks (this includes destructors).</li> If we cancel a parent task the child task should be cancelled too.</li> If a child task raises an error, the parent task should be able to catch it.</li> </ol> The benefit of this is that code like this will never have dangling tasks</em>, and will largely run in a predictable manner. This has all sorts of interesting properties, such as graceful shutdown trivially falling out of this model - but as I mentioned, that's something for a different post. The takeaway should be that "structured concurrency" is a desirable property for any concurrent code to have.</p> ^{1</sup> I've been meaning to for a while now, but alas. Time limits are a thing.</p> </div>}Dangling by default</h2> As we mentioned in the same post last time, by default most popular runtimes do not behave in a manner which is compatible with structured concurrency. When a JoinHandle</code> is dropped it isn't joined or cancelled, instead it's detached</em> causing it to dangle.</p> This behavior matches that of std::thread::JoinHandle</code> as well, but as Matklad remarked years ago</a> that behavior in threads is not without issues. In hindsight "join on drop" would have been a better default, but we can't easily change the behavior of std::thread</code> anymore. What we can</em> do is ensure that the behavior of asynchronous tasks default to the right thing. After all: modeling tasks after threads was a choice</em>, we can just as easily make a different choice. std::task</code> is also the one API we already don't expect to want to overload</a>, which means we're not as restricted in the API space as we are with other async stdlib APIs.</p> So if threads are not the right type to emulate the behavior of for tasks, what might be? I suspect the answer is Future</code>. The futures API RFC</a> mentions that cancelling futures should be done by dropping them. Which means that if our tasks were to emulate futures, we should have "cancel-on-drop" semantics too - as opposed to "join on drop" semantics.</p> There already exists a runtime which implements this in the wild: smol</code></a>, which is powered by the async-task</code></a> crate. "cancel-on-drop" semantics</a> mean that cancellation of an outer task will automatically propagate to inner tasks as well. Making it structurally concurrent by default.</p> async let</h2> "What if the behavior of futures and tasks were closer to each other"</em> has been an intrusive thought for me lately. Creating futures in Rust is easy, but spawning tasks is significantly harder. Why is that? It turns that the people working on Swift were pondering something similar not too long ago, and they've implemented a solution: async let</code></a>. Here's a basic example of async let</code> in action:</p> // given: </span>// func chopVegetables() async throws -> [Vegetables] </span>// func marinateMeat() async -> Meat </span>// func preheatOven(temperature: Int) async -> Oven </span> </span>func </span>makeDinner() async throws -> Meal { </span> async </span>let</span> veggies = chopVegetables() </span> async </span>let</span> meat = marinateMeat() </span> async </span>let</span> oven = preheatOven(temperature: </span>350</span>) </span> </span> </span>let</span> dish = Dish(ingredients: await [</span>try</span> veggies, meat]) </span> </span>return try</span> await oven.cook(dish, duration: .hours(</span>3</span>)) </span>} </span></code></pre> Each instance of async let</code> creates a Swift type equivalent to our Task</code> type in Rust. And awaiting an array of them is akin to our future::join</code> operations. Swift provides a global runtime out of the box, and awaiting one of these tasks schedules it on the runtime, and executes it in parallel. This is pretty neat, and compared to their TaskGroup</code> approach for concurrency, it significantly</em> reduces the amount of code needed for many uses 2</a></sup>.</p> ^2</sup>I didn't realize Swift had a way to concurrently await arrays. It seems we've independently come up with similar solutions</a>. That's probaly a good sign!</p> </div> What stands out to me is how easy this makes it to introduce ad-hoc parallelism into a codebase. The difference between parallel code and non-parallel code is the addition of a few async</code> keywords sprinkled in front of existing let</code> statements. The ease of this is really cool, and I think we can learn something from that!</p> Postfix spawn</h2> Now what if we could do this in Rust? Well, not literally</em> the same, but functionally</em>? What if we could make spawning tasks not much harder than creating futures? That'd be nice right? - and I'm happy to share I've got something working as part of the tasky crate</a>. Here's an example of it in use:</p> use </span>tasky::prelude::; </span> </span>let</span> res = async { "</span>nori is a horse</span>" }.</span>spawn</span>().await; </span>assert_eq!(res, "</span>nori is a horse</span>"); </span></code></pre> This will convert an existing future to a task, schedule it on the async-std</code> runtime, and await the result. But that's not all; when we call spawn</code> it doesn't actually create a task, it creates an async builder</a> which can be used to configure the task further before it's started. For example, we can use it to give our task a name before spawning it:</p> let</span> res = async { "</span>nori is a loaf of bread</span>" } </span> .</span>spawn</span>() </span> .</span>name</span>("</span>nori's tuna stash</span>".</span>into</span>()) </span> .await; </span>assert_eq!(res, "</span>nori is a loaf of bread</span>"); </span></code></pre> The way the "async builder" pattern works is by using the nightly IntoFuture</code> trait. Just like for item in iter</code> loops call IntoIterator</code> on iter</code>. So will .await</code> call IntoFuture</code> on whatever's being awaited. In this case: the TaskBuilder</code> type. This enables us to create chaining builders that are converted into futures when .await</code>ed.</p> Now what would Swift's earlier example look like if we translated it? For that we'd need the futures-concurrency</code></a> library too. Let's take a look:</p> // given: </span>// async fn chop_vegetables() throws -> io::Result<Vec<Vegetables>> </span>// async fn marinate_tofu() -> Tofu </span>// async fn preheat_oven(temperature: u16) -> Oven </span> </span>use </span>tasky::prelude::; </span>use </span>futures_concurrency::prelude::; </span>use </span>std::time::Duration; </span>use </span>std::io; </span> </span>async </span>fn </span>make_dinner</span>() -> io::Result<Meal> { </span> </span>let</span> veggies = </span>chop_vegetables</span>().</span>spawn</span>(); </span> </span>let</span> meat = </span>marinate_tofu</span>().</span>spawn</span>(); </span> </span>let</span> oven = </span>preheat_oven</span>(</span>350</span>).</span>spawn</span>(); </span> </span> </span>let</span> dish = Dish(&[veggies, meat].</span>try_join</span>().await?); </span> oven.</span>cook</span>(dish, Duration::from_secs(</span>60 </span>* </span>60 </span>* </span>3</span>)).await? </span>} </span></code></pre> There are still a few warts; but this gets us real</em> close to having semantics as smooth as Swift's. The most awkward pieces of the example are that we don't have Duration::from_hours</code>, and the array requires a try_join</code> method instead of implementing IntoFuture</code>. But other than that: it's quite close!</p> Converting the Swift example is nice to show that we can achieve parity with Swift, but it doesn't show how how much nicer postfix spawn is over the existing free functions. For that, let's take a look at what it looks like to spawn a longer chain of futures on a task:</p> // free-function, inline variant </span>let</span> body = task::spawn( </span> surf::get("</span>https://foo.bar</span>") </span> .</span>header</span>(some_header) </span> .</span>header</span>(other_header) </span> .</span>body_json</span>(data) </span> .recv_json::<MyResponse>() </span>).await; </span> </span>// free-function, variable variant </span>let</span> req = surf::get("</span>https://foo.bar</span>") </span> .</span>header</span>(some_header) </span> .</span>header</span>(other_header) </span> .</span>body_json</span>(data) </span> .recv_json::<MyResponse>(); </span>let</span> body = task::spawn(req).await; </span> </span>// postfix method </span>let</span> body = surf::get("</span>https://foo.bar</span>") </span> .</span>header</span>(some_header) </span> .</span>header</span>(other_header) </span> .</span>body_json</span>(data) </span> .recv_json::<MyResponse>() </span> .</span>spawn</span>() </span> .await; </span></code></pre> What we're not showing in this example are the extra imports free-functions will always require. Having a postfix spawn function would always just work as part of the Future</code> trait 3</a></sup>.</p> Having postfix syntax is not only easier to use, it also composes better with existing async features such as .await</code> and IntoFuture</code>. And removing minor speedbumps for common operations is a part of API design that should not be underestimated.</p> ^{3</sup> The working group is working on providing shared spawn</code> and Task</code> abstractions from the stdlib. This is very much an API we could provide.</p> </div>}Design Considerations</h2> The implementation of tasky</code>'s postfix Future::spawn</code> methods are not yet as polished as I'd like them to be. I knocked them out in about an hour last Friday, and I'm out with COVID right now so I don't see myself trying to polish them anytime soon. Though I do think it might be useful to note what would be done differently if I had the chance though. So let's take a look at the definition:</p> mod </span>sealed { </span> </span>/// Sealed trait to determine what type of bulider we got. </span> </span>pub trait </span>Kind {} </span>} </span> </span>/// A local builder. </span>#[</span>derive</span>(Debug)] </span>pub struct </span>Local; </span>impl </span>sealed::Kind </span>for </span>Local {} </span> </span>/// A nonlocal builder. </span>#[</span>derive</span>(Debug)] </span>pub struct </span>NonLocal; </span>impl </span>sealed::Kind </span>for </span>NonLocal {} </span> </span>/// Task builder that configures the settings of a new task. </span>pub struct </span>Builder<Fut: Future, K: sealed::Kind> { ... } </span>impl</span><Fut> IntoFuture </span>for </span>Builder<Fut, NonLocal> { ... } </span>impl</span><Fut> IntoFuture </span>for </span>Builder<Fut, Local> { ... } </span> </span>pub trait </span>FutureExt: Future + Sized { </span> </span>/// Spawn a task on a thread pool </span> </span>fn </span>spawn</span>(</span>self</span>) -> Builder<</span>Self</span>, NonLocal> </span> </span>where </span> </span>Self</span>: Send, </span> { </span> Builder { </span> kind: PhantomData, </span> future: </span>self</span>, </span> builder: async_std::task::Builder::new(), </span> } </span> } </span> </span> </span>/// Spawn a task on the same thread. </span> </span>fn </span>spawn_local</span>(</span>self</span>) -> Builder<</span>Self</span>, Local> { </span> Builder { </span> kind: PhantomData, </span> future: </span>self</span>, </span> builder: async_std::task::Builder::new(), </span> } </span> } </span>} </span></code></pre> We provide two methods on Future</code>: spawn</code> and spawn_local</code>. One spawns Send</code> futures, the other one !Send</code> futures 4</a></sup>. The builder needs to know what "mode" it's in to spawn the right future using IntoFuture</code>, so to do that we thread through a generic param which dictates the spawning mode.</p> The way we currently do this is not ideal. Enum variants aren't types</a> yet, so the best alternative we have is to emulate this using traits and manually written types. Though I think, ideally, we'd be able to differentiate based on a context</a> so that we can be generic over the backend. We don't have a notion yet of what runtime APIs would look like in the stdlib, so perhaps surfacing this as two different traits would actually work well.</p> ^4</sup>These are not the only spawning modes possible; I should write a post about a third mode (Send</code> once, future will not move threads after that) at some point. For now, check this out: raftario/async-into-future</a>.</p> </div> The other thing this API hasn't covered is the spawn_blocking</code></a> API. This convers synchronous operations to futures, backed by a thread pool. We could</em> have a postfix variant of this as well, but I'm not sure whether it's a good idea. In theory we could have a .spawn</code> method for any type T: !Future</code> using some sort of trait. But I'm not sure how much value that would provide over a free function. Personally I'm a bit wary of spawn_blocking</code>, and think if we we were to expose it from the stdlib, a free function would probably suffice.</p> Conclusion</h2> async-std</code> first proposed the model of: "What if tasks are async threads". smol</code> moved away from this by providing "cancel-on-drop" semantics. In this post we showed some of the benefits gained by moving further away from that model, and closer to: "What if tasks are parallelizable futures?"</p> In the future I'd very much like to explore whether we could move even closer to the "tasks-as-futures" model. Right now the tasky</code> crate allows people to break structured concurrency guarantees by calling JoinHandle::detach</code> or mem::forget(handle)</code>:</p> use </span>tasky::prelude::; </span> </span>async { </span>loop </span>{} } </span>// runs forever on a thread </span> .</span>spawn</span>() </span> .</span>into_future</span>() </span> .</span>detach</span>(); </span></code></pre> What if we made it so tasks aren't spawned on executors until .await</code>ed, just like futures? What if we did away with detach</code> entirely? We can't guarantee that all runtimes behave this way, since people can implement their own. But for the stdlib, would that make sense? This would still use "cancel on drop semantics" since .await</code> points mark cancellation sites</a>. But we'd also gain "tasks aren't started until .await</code>ed", just like futures. Just how much do we value having structured concurrency? Just what exactly does eager execution of tasks provide us right now?</p> Structured concurrency increases safety in concurrent programs, and I think it's exploring in closer detail what it provides, and what we need to do to achieve it. But until then: I hope you enjoyed this post! I wrote this because indexing a bunch of the Swift proposals, learned about async let</code>, and got excited. And I hope you're excited it now too!</p> Safe Pin Projections Through View Types Fri, 04 Mar 2022 00:00:00 +0000 "Pinning" is one of the harder concepts to wrap your head around when writing async Rust. The idea is that we can mark something as: "This won't change memory addresses from this point forward". This allows self-referential pointers to work, which are needed for async borrowing over await points. Though that's what it's useful for. In practice, how we use it is (generally) through one of three techniques:</p> stack pinning</strong> 1</a></sup>: this puts an object on the stack and ensures it doesn't move.</li> heap pinning</strong>: using Box::pin</code> to pin a type on the heap to ensure it doesn't move. This is often used as an alternative to stack pinning.</li> pin projections</strong>: convert from coarse: "This whole type is pinned" to more fine-grained: "Actually only these fields on the type need to be pinned".</li> </ul> ^{1</sup> Annoyingly that's inaccurate. We can have a future which we later box. That means we pin on the "stack" only relative to the function, even if the future itself may later live on the heap. "inline" vs "out of bounds" pinning may be more accurate. Though if you look at what a "stack" and "heap" are, you quickly find out those are made-up too. It's all stuff living somewhere, maybe. I find "stack-pinning" useful way to explain "pinned inside the function body", so let's just go with that for now, alright?</p> </div>}“stack pinning” is the less common of the two operations, but we’re on a path to adding first-class support to it via the core::pin::pin!</code> macro</a>. For “pin projection” we don’t have a path towards inclusion yet, and we’re left relying on ecosystem solutions such as the pin-project</code></a> crate2</a></sup> (see footnote for praise).</p> ^{2</sup> pin-project</code> is an excellent piece of engineering; I want there to be zero doubt about that. It's what makes the futures ecosystem possible, and has made it so that people (like myself) can safely</em> author futures by hand without risking undefined behavior. While this post seeks to explore ways in which we might move past requiring the need to use pin-project</code>, it's not because of issues with the crate. It's because I think there might be additional value in lifting this to the language level in a way that a crate cannot capture (simply by nature of being a crate).</p> </div> The status quo is not bad at all, but it’d be a shame if this is where we stopped. Pinning is useful for so much more than just futures, and perhaps by making it more accessible people could experiment with it more. In this post I want to share the beginnings of an idea I have for how we might be able to add pin projections as a first-class language feature to Rust.</p>}What are pin projections?</h2> As we mentioned: pin projections convert from coarse “this whole type is pinned” to more fine-grained: “Actually only these fields on the type need to be pinned”.</p> There’s a thorough explainer about this on the std::pin</code> docs in the stdlib</a>. But sometimes it’s better to show rather than tell. So let’s write a quick example using the pin-project</code> crate. Let’s create a sleep</code> future which uses some</em> internal timer (work with me here 3</a></sup>), and guarantees that it panics if it’s polled again after it’s completed:</p> ^{3</sup> Those familiar enough with runtime implementations might recognize this API as async-io</code>'s Timer</code> API. This type is Unpin</code>, so if we were actually using that here we didn't need to perform pin projections for things to work. But let's pretend that our timer here does</em> need to be pinned so we can write an example.</p> </div>}// Our main API which creates a future which wakes up after <dur> </span>pub fn </span>sleep</span>(</span>dur</span>: Duration) -> Sleep { </span> Sleep { </span> timer: Timer::after(dur), </span> completed: </span>false</span>, </span> } </span>} </span> </span>// Our future struct. The `timer` implements `Future`, </span>// and is the type we want to keep fixed in memory. </span>// But the `completed` field doesn't need to be pinned, </span>// so we don't mark it with "pin". </span>#[</span>pin_project</span>] </span>pub struct </span>Sleep { </span> #[</span>pin</span>] </span> </span>timer</span>: Timer, </span> </span>completed</span>: </span>bool</span>, </span>} </span> </span>impl </span>Future </span>for </span>Sleep { </span> </span>type </span>Output = Instant; </span> </span> </span>fn </span>poll</span>(</span>self</span>: Pin<&</span>mut Self</span>>, </span>cx</span>: &</span>mut </span>Context<'_>) -> Poll<</span>Self::</span>Output> { </span> assert!(!</span>self</span>.completed, "</span>future polled after completing</span>"); </span> </span> </span>// This is where the pin projection happens. We go from a `Pin<&mut Self>` </span> </span>// to a new type which can be thought of as: </span> </span>// `{ timer: Pin<&mut Timer, completed: bool }` </span> </span>let</span> this = </span>self</span>.</span>project</span>(); </span> </span>match</span> this.timer.</span>poll</span>(cx) { </span> Poll::Ready(instant) => { </span> this.completed = </span>true</span>; </span> Poll::Ready(instant.</span>into</span>()) </span> } </span> Poll::Pending => Poll::Pending, </span> } </span> } </span>} </span></code></pre> If you haven’t seen any manual futures code before, that can be a lot to take in. It’s a goal of the Rust Async WG to make it so people need to write as little Pin</code> code as possible to use async Rust. So if this overwhelming to you / you’re unsure what’s going on. Please don’t feel pressured you need to learn this.</p> But for folks who are curious: the interesting part here is the call to self.project()</code>. This method is provided by the pin-project</code> crate, and allows us to convert from Pin<&mut Self></code> to a generated type where only certain fields are pinned. I like to visualize it for myself like this:</p> // before projecting </span>self</span>: Pin<&</span>mut</span> Sleep </span>{</span> timer: Timer, completed: </span>bool</span> }> </span> </span>// after projecting </span>self</span>: &</span>mut</span> Sleep { timer: Pin<&</span>mut</span> Timer>, completed: </span>bool </span>}> </span></code></pre> Both provide the guarantee that types which need to be pinned in fact are</em> pinned. But the “after projecting” example does so on a per-field basis, rather than locking the whole thing in-place. And if the whole struct is locked in place, it means we can’t mutate fields on it. Which in turn means we could never mark completed</code> as true</code>. So we have</em> to project to write stateful futures.</p> View Types</h2> In their post “view types for Rust”</a> Niko talks about a new language construct called “view types” which can create a finer-grained interface for borrowing values. The whole post is worth a read, but to recap the example:</p> // This is the code we start with </span> </span>struct </span>WonkaShipmentManifest { </span> </span>bars</span>: Vec<ChocolateBar>, </span> </span>golden_tickets</span>: Vec<</span>usize</span>>, </span>} </span> </span>impl </span>WonkaShipmentManifest { </span> </span>fn </span>should_insert_ticket</span>(&</span>self</span>, </span>index</span>: </span>usize</span>) -> </span>bool </span>{ </span> </span>self</span>.golden_tickets.</span>contains</span>(&index) </span> } </span>} </span> </span>impl </span>WonkaShipmentManifest { </span> </span>fn </span>prepare_shipment</span>(</span>self</span>) -> Vec<WrappedChocolateBar> { </span> </span>let mut</span> result = vec![]; </span> </span>for </span>(bar, i) in </span>self</span>.bars.</span>into_iter</span>().</span>zip</span>(</span>0</span>..) { </span> </span>let</span> opt_ticket = </span>if </span>self</span>.</span>should_insert_ticket</span>(i) { </span> Some(GoldenTicket::new()) </span> } </span>else </span>{ </span> None </span> }; </span> result.</span>push</span>(bar.</span>into_wrapped</span>(opt_ticket)); </span> } </span> result </span> } </span>} </span></code></pre> And it yields this error:</p> error[E0382]: borrow of partially moved value: `self` </span> --> src/lib.rs:16:33 </span> \| </span>15 \| for (bar, i) in self.bars.into_iter().zip(0..) { </span> \| ----------- `self.bars` partially moved due to this method call </span>16 \| let opt_ticket = if self.should_insert_ticket(i) { </span> \| ^^^^ value borrowed here after partial move </span></code></pre> However if we change the definition of should_insert_ticket</code> to be more fine-grained, we can tell the compiler that our borrows do not in fact overlap, which makes the borrow checker happy.</p> impl </span>WonkaChocolateFactory { </span> </span>fn </span>should_insert_ticket</span>(&{ golden_tickets } </span>self</span>, </span>index</span>: </span>usize</span>) -> </span>bool </span>{ </span> </span>// ^ this is the "view type" </span> </span>self</span>.golden_tickets.</span>contains</span>(&index) </span> } </span>} </span></code></pre> To me the core insight of “view types” is that by encoding more details in type signatures, the compiler is able to make better decisions about what’s allowed or not. Who’d have guessed?</p> View types for pin projection</h2> So far we’ve looked at “pin projection” and “view types” individually. But what happens if we put them together? Could we express pin projections using</em> view types? I think we might! “view types” as proposed by Niko are a way to teach the compiler about lifetimes in finer detail. I think we could apply the same logic to use view types to teach the compiler about pinning in finer detail too.</p> ⚠ So before we dive in I want to remind folks again that this is just an idea. I’m not on the lang team. I am in not in any position to make decisions about this. I also don't consider myself an expert in pin semantics, so I might have gotten things wrong. I am not saying that we should prioritize work on this. I'm sharing thoughts here in the hopes of, well, progressing the language. And sometimes putting half-baked ideas into writing can help others refine them into something practical later on.</em> ⚠</p> Anyway, let’s take our earlier example and try and fit pin projections on it using view types. Taking it very literally we could imagine the signature of Future::poll</code> to be possible to express like this:</p> // base </span>fn </span>poll</span>(</span>self</span>: Pin<&</span>mut Self</span>>, </span>cx</span>: &</span>mut </span>Context<'_>) -> Poll<</span>Self::</span>Output> { .. } </span> </span>// using view types. `timer` is pinned. </span>fn </span>poll</span>(&</span>mut</span>{ timer, completed } </span>self</span>: Pin<&</span>mut Self</span>>, </span>cx</span>: &</span>mut </span>Context<'_>) -> Poll<</span>Self::</span>Output> { .. } </span></code></pre> Okay okay, this is a super literal application of “what if view types would just do the pin projection”. But bear with me, we’ll go over all details in a second. For now let’s just assume this works, and slot it into the earlier example:</p> pub fn </span>sleep</span>(</span>dur</span>: Duration) -> Sleep { .. } </span> </span>pub struct </span>Sleep { </span> </span>timer</span>: Timer, </span> </span>completed</span>: </span>bool</span>, </span>} </span> </span>impl </span>Future </span>for </span>Sleep { </span> </span>type </span>Output = Instant; </span> </span> </span>fn </span>poll</span>(&</span>mut</span>{ timer, completed } </span>self</span>: Pin<&</span>mut Self</span>>, </span>cx</span>: &</span>mut </span>Context<'_>) -> Poll<</span>Self::</span>Output> { </span> </span>// ^ we're using the view type here </span> assert!(!completed, "</span>future polled after completing</span>"); </span> </span>match</span> timer.</span>poll</span>(cx) { </span> Poll::Ready(instant) => { </span> completed = </span>true</span>; </span> Poll::Ready(instant.</span>into</span>()) </span> } </span> Poll::Pending => Poll::Pending, </span> } </span> } </span>} </span></code></pre> I think overall this gets close to the feel we want pin projections to have. Instead of our base type we narrow it down into its parts. But it’s probably not everything we want to do yet, and it certainly doesn’t cover all aspects of pin projections yet.</p> Ergonomics</h2> In my opinion the Future::poll</code> signature above is rather noisy; the view type might have helped simplify the function body and struct declration, but it's definitely not helped the function signature. And somewhat annoyingly: we don't have any more information than we started with. We can't actually tell which fields are pinned!</p> fn </span>poll</span>(&</span>mut</span>{ timer, completed } </span>self</span>: Pin<&</span>mut Self</span>>, </span>cx</span>: &</span>mut </span>Context<'_>) -> Poll<</span>Self::</span>Output> { </span></code></pre> Perhaps the compiler can be clever enough to infer that all !Unpin</code> types in a view type should remain pinned when viewed. But even if it can, type signatures are as much for humans as they are for the compiler. And this tells us very little about what is pinned. Instead I think it’d be clearer if this type of lowering required contextual pin</code> annotations:</p> fn </span>poll</span>(&</span>mut</span>{ pin timer, completed } </span>self</span>: Pin<&</span>mut Self</span>>, </span>cx</span>: &</span>mut </span>Context<'_>) -> Poll<</span>Self::</span>Output> { </span></code></pre> Now we can tell that the timer</code> is pinned, but completed</code> is not. We finally have all the detail we need, but the signature is even more overwhelming than it was. I think it's worth asking at this point whether there's anything we could remove. If we look at the projected type, we can see there are three parts to it which tell us what "self" is supposed to look like:</p> &</span>mut </span>{ pin timer, completed } </span>self</span>: Pin<&</span>mut Self</span>> </span>// ^ ^ ^ </span>// \| \| This only exists if `self` is this type. </span>// \| This is about a `self` param. </span>// We have a mutable view type with these fields. </span></code></pre> There are three parts here, which if we assume we can remove one, we can have three different permutations:</p> // view type + self </span>fn </span>poll</span>(&</span>mut</span>{ pin timer, completed } </span>self</span>, </span>cx</span>: &</span>mut </span>Context<'_>) -> Poll<</span>Self::</span>Output> { </span>// ^ no right-hand `Pin<&mut Self>` </span> </span>// view type + self </span>fn </span>poll</span>(&</span>mut</span>{ pin timer, completed } pin </span>self</span>, </span>cx</span>: &</span>mut </span>Context<'_>) -> Poll<</span>Self::</span>Output> { </span>// ^ `Pin<&mut Self>` replaced by `pin self` </span> </span>// view type + arbitrary Self-type </span>fn </span>poll</span>(&</span>mut</span>{ pin timer, completed }: Pin<&</span>mut Self</span>>, </span>cx</span>: &</span>mut </span>Context<'_>) -> Poll<</span>Self::</span>Output> { </span>// ^ no left-hand `self` type </span> </span>// self + arbitrary Self-type </span>fn </span>poll</span>(</span>self</span>: Pin<&</span>mut Self</span>>, </span>cx</span>: &</span>mut </span>Context<'_>) -> Poll<</span>Self::</span>Output> { </span>// ... no view type :( </span></code></pre> None of these feel particularly great, but they do provide us with some insight. Pin<&mut Self> can be useful to know _when_ this implementation is available. </code>self` is actually never used since we immediately destructure, but it is useful to know that this is a method and not a free-function. The view type is what makes pin projections possible, so we want to keep that somehow.</p> I'm not sure which attributes make most sense to highlight. The original design keeps all properties, but feels overwhelming. On reviewing this post, my co-worker Eric suggested we try and take "view types" proposal, performing destructuring like we do in patterns, dropping the left-hand variable declaration entirely:</p> // self + arbitrary Self-type + view type </span>fn </span>poll</span>(</span>self</span>: Pin<&</span>mut Self </span>{</span> pin timer, completed }>, </span>cx</span>: &</span>mut </span>Context<'_>) -> Poll<</span>Self::</span>Output> { </span> </span>// self + pin Self-type + view type </span>fn </span>poll</span>(</span>self</span>: pin </span>Self</span> { pin timer, completed }, </span>cx</span>: &</span>mut </span>Context<'_>) -> Poll<</span>Self::</span>Output> { </span> </span>// view type + self </span>fn </span>poll</span>(</span>self</span>: &</span>mut Self</span> { pin timer, completed }, </span>cx</span>: &</span>mut </span>Context<'_>) -> Poll<</span>Self::</span>Output> { </span> </span>// view type </span>fn </span>poll</span>(&</span>mut Self</span> { pin timer, completed }, </span>cx</span>: &</span>mut </span>Context<'_>) -> Poll<</span>Self::</span>Output> { </span></code></pre> I think we'll need to keep the self</code> param to not create confusion with Self</code> methods</a> 4</a></sup>. But it's hard to say which option out of these would be the best. The second one is probably the simplest. But the first one conveys more information. Luckily we don't need to make any decisions now, so we can just settle for having looked at the space a little.</p> ^{4</sup> I honestly don't understand why this isn't a concrete method on Arc</code>. Is this so we don't accidentally hit the Deref</code> implementation? Perhaps a compiler limitation? I don't really get it, heh.</p> </div>}Pin projections and drop</h2> Something we haven’t covered yet is that when pin projecting there are Drop</code> requirements. Put bluntly, the signature of std::drop::Drop</code> is wrong</em>. If we’d known about “pinning” before Rust 1.0 we likely would've designed the Drop</code> trait differently.</p> The reason why Drop</code> and pinning conflict is because once something has been pinned it should pinned forever after 5</a></sup>. But Drop</code> takes &mut self</code>, even if a value has previously been pinned</em>, violating this requirement. And so when we implement Drop</code> for our async types we have to make sure to always convert &mut self</code> back to self: Pin<&mut Self></code></a>:</p> ^{5</sup> Unless it's unpinned. But "unpin" is basically: "I don't care if I'm pinned or not, just do whatever", so let's pretend we're not talking about it here either.</p> </div>}impl </span>Drop </span>for </span>Type { </span> </span>fn </span>drop</span>(&</span>mut </span>self</span>) { </span> </span>// `new_unchecked` is okay because we know this value is never used </span> </span>// again after being dropped. </span> </span>inner_drop</span>(</span>unsafe </span>{ Pin::new_unchecked(</span>self</span>)}); </span> </span>fn </span>inner_drop</span>(</span>this</span>: Pin<&</span>mut</span> Type>) { </span> </span>// Actual drop code goes here. </span> } </span> } </span>} </span></code></pre> The pin-project</code> crate covers this case for us. It disallows implementing Drop</code> manually, and requires we implement pin_project::PinnedDrop</code></a> instead, which has the right signature:</p> use </span>std::pin::Pin; </span>use </span>pin_project::{pin_project, pinned_drop}; </span> </span>#[</span>pin_project</span>(PinnedDrop)] </span>struct </span>PrintOnDrop { </span> #[</span>pin</span>] </span> </span>field</span>: </span>u8</span>, </span>} </span> </span>#[</span>pinned_drop</span>] </span>impl </span>PinnedDrop </span>for </span>PrintOnDrop { </span> </span>fn </span>drop</span>(</span>self</span>: Pin<&</span>mut Self</span>>) { </span> println!("</span>Dropping: </span>{}</span>", </span>self</span>.field); </span> } </span>} </span> </span>fn </span>main</span>() { </span> </span>let</span> _x = PrintOnDrop { field: </span>50 </span>}; </span>} </span></code></pre> If we want to make pinning more accessible, then we cannot just skip over the interactions with Drop</code>. I'm not sure whether this allows us to trigger soundness bugs using purely safe Rust - but I suspect the answer to that is likely irrelevant. It's really easy to get lost in technicalities that we lose sight of the wider point: Pinning has too many gotchas to use without pin-project</code>. And it's hard to tease apart to which degree the issues are with pinning as a concept, or mistakes we've made in the past in the language.</p> Perhaps "mistakes" sounds too harsh. After all, Pin</code> was introduced after Rust 1.0, so how could we have gotten this right? And while that's true, I think that perspective is a bit of a dead-end. If the issues are with Pin</code>, then logically we'll seek to fix Pin</code>. And realizing we're unable to fix Pin</code>, we'll naturally seek to limit the usage of it. But rather than thinking about pin as "introducing issues into Rust", what if we instead think of pin as "surfacing issues we have in Rust".</strong> Pin is a fundamental type in Rust. Transformative even. It's worth pondering what we could do with it if it was better integrated into the language.</p> Maybe we should bite the bullet and actually fix</em> our Drop</code> impl. The main reason why this hasn't been done is because it's backwards incompatible 6</a></sup>. But likely the most important reason after that is because it would grossly complicate the ergonomics of implementing Drop</code> for people. Perhaps that calculation changes if the ergonomics or pinning were less bad. Say we took one of the ergo examples and applied it to our earlier drop example:</p> ^{6</sup> As my colleague Eric pointed out, one way we could replace Drop</code> with a new trait in a backwards-compatible manner is through blanket impls. We could define PinnedDrop</code>, and provide a blanket impl of PinnedDrop</code> for T: Drop + Unpin</code>. Then we change the compiler to desugar drops to a pin and a call to PinnedDrop::drop</code> instead. Though replacing Drop</code> with something else (even for something as important as soundness) might be tricky to coordinate and communicate.</p> </div>}struct </span>PrintOnDrop { </span> </span>field</span>: </span>u8</span>, </span>} </span> </span>impl </span>Drop </span>for </span>PrintOnDrop { </span> </span>// We've dropped `self: Pin<&mut Self>` from our signature </span> </span>// because we immediately destructure it into its parts </span> </span>// inside the type signature anyway. </span> </span>fn </span>drop</span>(</span>self</span>: pin </span>Self</span> { pin field }) { </span> println!("</span>Dropping: </span>{}</span>", field); </span> } </span>} </span> </span>fn </span>main</span>() { </span> </span>let</span> _x = PrintOnDrop { field: </span>50 </span>}; </span>} </span></code></pre> I don't think this looks noticably worse than today's drop trait. And hold onto your tin-foil caps, I think this could possibly hint at another set of optimizations we might be able to perform. What if for structs where all types are Unpin</code> we could create whole-sale projections. u8</code> here doesn't actually need to be pinned, it's just for show. So what if we actually were able to write that:</p> struct </span>PrintOnDrop { </span> </span>field</span>: </span>u8</span>, </span>} </span> </span>impl </span>Drop </span>for </span>PrintOnDrop { </span> </span>// This is a short-hand for a destructuring of `Pin<&mut Self>` </span> </span>// where all fields are `Unpin`, requiring no pinning and thus </span> </span>// no destructuring. </span> </span>fn </span>drop</span>(&</span>mut </span>self</span>) { </span> println!("</span>Dropping: </span>{}</span>", </span>self</span>.field); </span> } </span>} </span> </span>fn </span>main</span>() { </span> </span>let</span> _x = PrintOnDrop { field: </span>50 </span>}; </span>} </span></code></pre> destructuring can be nice. That would allow us to write &mut self</code> as sugar for self: Pin<&mut Self> where Self: Unpin</code>. Meaning: for all sound</em> implementations of Drop</code> everything would continue working as they do today.</strong> The only difference is that the trait declaration of Drop</code> would change. And we'd catch potential unsound invocations of Drop</code>, requiring them to be sound at the function signature level. This includes pin-project</code>'s use as well, but we'd likely be able to coordinate roll-out, deprecation, etc. in the crate.</p> This is not unlike how &mut self</code> is syntactic sugar for &mut self: &mut Self</code> already. We could make it so that we can lower Pin<&mut Self></code> to Self</code> if it's always safe to do so. Anyway. That's probably enough speculation for this section. To close it off, so far we haven't yet figured out how to introduce async Drop</code> into the language. One option mentioned so far been to extend Drop</code> with poll</code> methods</a>. However way we go about this, we'll have to be looking more closely at Drop</code> and Pin</code> soon anyway.</p> #[repr(packed)]</h2> There are other considerations such as not being able to pin a #[repr(packed)]</code> type. And honestly: I’m not super sure how we can fix all of this. Maybe we should raise #[repr(packed)]</code> into the type-system and make Pin<T> where T: !marker::Packed</code>. Or perhaps pin projections should magick their way into knowing not to do this. I'm not sure. But this is something to figure out if we want to remove the unsafe {}</code> marker for pin projections.</p> Unpin</h2> So far we haven't talked about Unpin</code> at all. I mean, unless you've read the footnotes. But I assume that's nobody, so here's a recap: Unpin</code> means that a type simply doesn't care about Pin</code> at all. Moving it around in memory won't cause it to spontaniously combust, so we can just move it in and out of Pin</code> wrappers whenever we like. This is true for most types ever created. Only types which are self-referential (hold pointers to fields contained within itself) are marked as !Unpin</code>. Which is true for many futures created using async {}</code>, because that's what borrows across .await</code> points desugar into.</p> Unpin</code> is an auto-trait: meaning it's implemented for a type if all the fields of the type implement it. But unlike auto-trait such as Send</code> and Sync</code>, it's entirely safe</em> to implement. And this causes some issues.</p> This means that even if our struct contains types which are !Unpin</code>, we can still manually implement Unpin</code> on it:</p> use </span>std::marker::PhantomPinned; </span>use </span>std::marker::Unpin; </span> </span>// `Foo` holds an `!Unpin` type, which makes </span>// `Foo: !Unpin`. </span>struct </span>Foo { </span> </span>_marker</span>: PhantomPinned </span>} </span> </span>// Oh nevermind, we can just mark it as `Unpin` anyway. </span>impl </span>Unpin </span>for </span>Foo {} </span></code></pre> I believe the reasoning is that since "pin projections" require unsafe</code>, upholding the validity of Unpin</code> should just be done there. But that's an issue if we're looking for ways to actually make pin projections safe to perform. Upholding invariants shouldn't be done at the projection site; it needs to be done on declaration!</p> The pin-project</code> crate works around this by disallowing safe implementations of Unpin</code> and instead introducing their own UnsafeUnpin</code> trait</a>. This flips the safety requirement back to the Unpin</code> implementation, requiring invariants are upheld by the implementer of the trait.</p> use </span>pin_project::{pin_project, UnsafeUnpin}; </span> </span>#[</span>pin_project</span>(UnsafeUnpin)] </span>struct </span>Struct<K, V> { </span> #[</span>pin</span>] </span> </span>field_1</span>: K, </span> </span>field_2</span>: V, </span>} </span> </span>// Implementing `Unpin` is unsafe here. </span>unsafe impl</span><K, V> UnsafeUnpin </span>for </span>Struct<K, V> </span>where</span> K: Unpin + Clone {} </span> </span>// Which means calling `struct.project()` later on can be safe. </span></code></pre> In order for pin projections to be safe, manual implementations of Unpin</code> need to be unsafe</code>. This is because someone needs to be responsible for correctly implementing the invariants; and it either needs to be where we perform the projection, or where we declare types as Unpin</code>. "both" is annoying. "neither" is impossible 7</a></sup>.</p> ^{7</sup> We need to be able to label types as Unpin</code> somehow. Even if every type in the stdlib implements it, the stdlib itself is not magic. And in the ecosystem too: core types require being able to be marked with it.</p> </div> It seems that if we want to make pin projections a first-class construct in Rust, we'll have to mark Unpin</code> as unsafe</code>. Annoyingly this would be a change to the stdlib, and we don't have a process like "editions" where we can flip a switch on changes like these. At least not yet.</p> Making this happen wouldn't be easy. And we'd need to assess whether the impact justifies going down this path. But maybe it does. And maybe we should.</p>}Pin keyword in other places?</h2> At the start of this post we mentioned that stack-pinning is gaining first-class support in Rust via the core::pin::pin!</code> macro</a>. But later on in the post we also discuss the possibility of improving pin projection semantics by introducing a pin</code> keyword in the view type. It's worth asking: could a pin</code> keyword be useful in other places too?</p> I think, maybe? Even without view types, we could imagine all function signatures which currently take self: Pin<&mut Self></code> could probably be simplified to take pin self</code> instead:</p> // Before </span>pub trait </span>Future { </span> </span>type </span>Output; </span> </span>fn </span>poll</span>(</span>self</span>: Pin<&</span>mut Self</span>>, </span>cx</span>: &</span>mut </span>Context<'_>) -> Poll<</span>Self::</span>Output>; </span> </span>// ^ Using an "arbitrary self type" </span>} </span> </span>// After </span>pub trait </span>Future { </span> </span>type </span>Output; </span> </span>fn </span>poll</span>(</span>self</span>: pin </span>Self</span>, </span>cx</span>: &</span>mut </span>Context<'_>) -> Poll<</span>Self::</span>Output>; </span> </span>// ^ Replaced by a `pin` keyword. </span>} </span></code></pre> Additionally, for stack pinning we could imagine we could replace the use of core::pin::pin!</code> with pin-bindings:</p> // Before </span>fn </span>block_on</span><T>(</span>fut</span>: impl Future<Output = T>) -> T { </span> </span>let mut</span> fut = pin::pin!(fut); </span> ... </span>} </span> </span>// Using `let pin` bindings in the function body. </span>fn </span>block_on</span><T>(</span>fut</span>: impl Future<Output = T>) -> T { </span> </span>let</span> pin fut = fut; </span> ... </span>} </span> </span>// Using `pin` in the function signature. </span>fn </span>block_on</span><T>(pin </span>fut</span>: impl Future<Output = T>) -> T { </span> ... </span>} </span></code></pre> I don't think this looks particularly out of place. It might be nice even. But the real question to ask is: is this useful? We have so many things we could be working on that we have to choose what to prioritize. Right now "pin" is probably misunderstood and under-used because it doesn't play well with the language. But if we fixed those issues, would we see people use pin</code> for more things 8</a></sup>? It's a bit of a chicken/egg problem; though we do need to have some</em> an answer in order to justify spending time on this.</p> ^{8</sup> As we mentioned: Pin</code> is what makes borrowing over .await</code> points work by allowing us to define self-referential structs. But outside of .await</code> we cannot yet define self-referential structs. If pinning is required to make that work, maybe there's reason to improve the ergonomics? I'm genuinely unsure; but I think that's the direction we should be asking questions in.</p> </div>}Conclusion</h2> In this post I’ve shown what pin projections are, what view types are, how the two could be combined, ergonomics considerations, possible interactions with Drop</code>, and interactions with Unpin</code>. In general it seems there are 3 questions to answer?:</p> Can we use view types to create pin projections? What should that look like?</li> How can we fix the signature of Drop</code>?</li> How can we lift #[repr(Packed)]</code> into the type system so we can disallow it to be pinned?</li> If we want to make pin projections safe, can we mark Unpin</code> as unsafe</code>?</li> </ul> Have I missed anything? Are there any other parts to consider? I’m not sure. I also want to point out that we don’t need a single unifying answer to solve all of these questions. It may be that each item has a different solution; but once we solve all of them we can start to realize safe pin projections at the lang level. As I’ve mentioned a few times now, I’d be keen to hear from others.</p> Anyway, this is just some stuff I started thinking of after having read Niko’s view types blog post midway through writing a bunch of futures by hand. I’m not sure whether delving deeper into this idea is worth it at all right now. As I mentioned pinning has potential</em>, but we’re trying hard to make users interact with it as little as possible. Kind of like how we have unsafe</code>, but we don’t really want people interacting with it much if we can help it, even if the stakes are different.</p> From what I’ve heard the Linux kernel folks are making use of pinning for some of the things they're doing already. I'm not sure what they're doing it for, but it’d be fascinating</em> to learn why. If people are thinking of improving the ergonomics of unsafe</code> to make it less hard to use. Perhaps it’d be worth to do the same for pinning as well?</p> Thanks to Ryan Levick</a> and Eric Holk</a> for helping review and correct this post prior to publishing. Niko Matsakis</a> for entertaining my delirious ramblings about this on Zulip a few days ago. And Boats</a>, with who I'm pretty sure I've discussed something like this before, even though time is a lie and at this point I no longer remember how long ago that may have been.</em></p> More Enum Types Tue, 15 Feb 2022 00:00:00 +0000 Yesterday I was talking with folks on Zulip about my last blog post</a>, and specifically about the potential ergonomic improvements “anonymous enums” could provide. In my post we only really looked at anonymous enums as a potential ergonomics improvement for Stream::merge</code>. But in this post I instead want to dig ever so slightly deeper and show how symmetry</em> is missing in the Rust language between struct and enum variants.</p> So I'm writing this post to dig into that some more. We'll be keeping it brief, focusing mostly on showing something which stood out to me. The intent is less so for me to be making recommendations about the direction the language should take, and mostly on walking you through my observations. There are times when I spend months performing rigorous analysis of things, and there’s times like these when I just want to share some thoughts. This post also serves as practice for me to edit less and publish faster.</p> So with all that out of the way, let’s dig in by looking at structs first.</p> Structs</h2> Structs are Rust’s most common data container. They come in three flavors: empty (no fields), record (key-value), or tuple (indexed fields). All three roughly work the same, and for the sake of simplicity we’ll only be looking at tuple structs in this post. So let’s start defining one!</p> #[</span>derive</span>(Debug, Copy, Clone, PartialEq, Eq, PartialOrd, Ord)] </span>struct </span>MyStruct(</span>u32</span>, </span>u16</span>); </span></code></pre> That’s right, that’s a tuple struct. It has two fields, both of which must be populated. And it has a bunch of derives, which allow us to perform common, useful operations on it. Now let’s create one and return it from a function.</p> #[</span>derive</span>(Debug, Copy, Clone, PartialEq, Eq, PartialOrd, Ord)] </span>struct </span>MyStruct(</span>u32</span>, </span>u16</span>); </span> </span>// named type </span>fn </span>foo</span>() -> MyStruct { </span> MyStruct(</span>100</span>, </span>12</span>) </span>} </span></code></pre> If you’ve ever used a tuple-struct that should look pretty familiar, right? We created an instance of the type, and returned it from the function. But you might be noticing that this is actually quite a bit of boilerplate. IDEs can help us with the derives. But they can’t help us work around defining the type, the layout, and naming it. What if we didn’t actually care about the name? Say, if this was just a helper function in our file? For cases where naming the returned types isn’t useful, we can use tuples:</p> // anonymous type </span>fn </span>foo</span>() -> (</span>u32</span>, </span>u16</span>) { </span> (</span>100</span>, </span>12</span>) </span>} </span></code></pre> This does almost the exact same thing as the code we had before, but it’s now much shorter! Gone is the struct declaration. And gone are the derives. This works because tuples have generic impls on them which say: “if all of my members implement a trait, then so will I.” You can almost think of it as automatically deriving traits, while named structs need to declare which traits they want to derive.</p> Now finally, sometimes we don’t want to expose which type we return. Often because we don’t want to provide any stability guarantees about the type we return. We want to be able to say: “the type we return conforms to this interface, but that’s all I’m telling you.” For example -> impl Future</code> was pretty common to author before we had async/.await</code> (in fact async {}</code> generates an impl Future</code> type, also known as “anonymous future”). But we can write this in our own code like so:</p> // type erased type </span>fn </span>foo</span>() -> impl Debug { </span> MyStruct(</span>100</span>, </span>12</span>) </span>} </span></code></pre> All this tells us is: “the type we’re returning here implements Debug</code>, and that’s it”.</p> Enums</h3> Now let’s take a look at enums. Much like structs, enums can carry values as well (empty, record, tuple) - but instead of being AND (contain multiple values), they give us XOR (contain an exclusive, single value) 1</a></sup>. Not sure how much sense this explainer makes, but please pretend it does. In an enum, only one variant is active at any given time. So let’s define an enum where we either have a u32</code> or a u16</code>:</p> ^{1</sup> Yes, "record enums" give us XOR and then AND. But those are almost like a different kind of shorthand for having named record fields nested in enums. If we have to go into those we'll never finish this post. So let's not think about them too much for now.</p> </div>}#[</span>derive</span>(Debug, Copy, Clone, PartialEq, Eq, PartialOrd, Ord)] </span>enum </span>MyEnum { </span> Left(</span>u32</span>), </span> Right(</span>u16</span>), </span>} </span></code></pre> The way we return an enum from a function is by constructing a variant. This is often times done based on a condition. So let’s pretend we have a function which is given a condition, and computes which enum to return:</p> #[</span>derive</span>(Debug, Copy, Clone, PartialEq, Eq, PartialOrd, Ord)] </span>enum </span>MyEnum { </span> Left(</span>u32</span>), </span> Right(</span>u16</span>), </span>} </span> </span>// named type </span>fn </span>foo</span>(</span>cond</span>: </span>bool</span>) -> MyEnum { </span> </span>if</span> cond { MyEnum::Left(</span>100</span>) } </span>else </span>{ MyEnum::Right(</span>12</span>) } </span>} </span></code></pre> This will either return the number 100u32</code> or the number 12u16</code>. That makes sense right?</p> Anonymous enums</h2> Now what if we wanted to simplify this, say if we were writing a quick helper. The boilerplate of defining an enum and derives might be distracting, and much like a tuple we just want to send some data along. We might imagine that we would want to write this:</p> // named type </span>fn </span>foo</span>(</span>cond</span>: </span>bool</span>) -> </span>u32 </span>\| </span>u16 </span>{ </span>// made up syntax for u32 OR u16 </span> </span>if</span> cond { </span>100 </span>} </span>else </span>{ </span>12 </span>} </span>} </span></code></pre> However, Rust doesn’t support this. Unlike “anonymous structs” (tuples), there is no such thing as an “anonymous enum”. There’s an open issue about it</a>, but I don’t know anyone who’s working on it. And I don’t know of any crates which provide implementations of this either.</p> Something which makes it difficult to prototype “anonymous enums” is that the syntax comes in pairs: not only do we need to figure out how to define them in function return positions, we need to think about how to use them as well. The version I’ve seen which I like the most used type ascription when matching, but that’s hard</em> to implement, and has a bunch of other lang implications as well:</p> match </span>foo</span>(cond) { </span> n: </span>u32 </span>=> {}, </span> n: </span>u16 </span>=> {} </span>} </span></code></pre> Though if someone has ideas how to prototype this, it would be fun to see what people can come up with.</p> Probably the closest we can get to “anonymous enums” is by using the either</a> crate from the ecosystem. I believe once upon a time this crate lived in the stdlib (pre 1.0), and it provides us with a Result</code>-like type which holds two generic params: one for the left-hand variant, and one for the right-hand variant. Much like “anonymous enums”, it allows us to bypass having to provide our own enum by having a pre-made one.</p> use </span>either::Either; </span> </span>// named type </span>fn </span>foo</span>(</span>cond</span>: </span>bool</span>) -> Either<</span>u32</span>, </span>u16</span>> { </span> </span>if</span> cond { Either::Left(</span>100</span>) } </span>else </span>{ Either::Right(</span>12</span>) } </span>} </span></code></pre> In the wild, types such as futures::future::Select</code></a> yield Either</code> variants when wanting to return one of two types. This works alright when joining two different futures, but when attempting to join more than two, you’ll end up with nested Join<Join<First, Second>, Third></code>, etc constructions which are not fun to work with.</p> In my opinion it would be better if Future::select</code>/ Future::race</code> would return a single type T</code>, allow race</code> to operate over N futures concurrently</a>, and make enum construction an explicit step. That way if Rust ever gets anonymous enums, the compiler could inference we’re passing different types, which can be unified using an anonymous enum.</p> let</span> a = future::ready(</span>1</span>u8</span>); </span>let</span> b = future::ready("</span>hello</span>"); </span>let</span> c = future::ready(</span>3</span>u16</span>); </span> </span>// The compiler could infer the Future's return type T </span>// should be an anonymous enum, which we can subsequently </span>// match over </span>match </span>(a, b, c).</span>race</span>().await { </span> n: </span>u8 </span>=> {}, </span> n: </span>u16 </span>=> {}, </span> n: &</span>str </span>=> {}, </span>} </span></code></pre> In async-std</code> the Future::race</code> method</a> already returns a single type T</code> instead of an Either</code> type. And for futures-concurrency</code></a> we’re likely going to do the same 2</a></sup>, but instead of providing a race</code> method from Future</code>, provide a Race</code> trait directly from container types (though we’re likely going to rename the traits before we merge them. Current contender: first/First</code> method/trait).</p> ^{2</sup> The plan is to merge futures-concurrency</code> back into async-std</code> once we figure out all the parts of it. But that's been taking a minute, and will likely take a bit longer. But it's important we get this right, sooooo. Hence the last 3 years of work.</p> </div>}Type-erased enums</h2> Now, taking things one level of abstraction further: what if we have N values, but we only care about the shape of the values returned. We could imagine we could write something like this:</p> // type erased type (currently unsupported) </span>fn </span>foo</span>(</span>cond</span>: </span>bool</span>) -> impl Debug { </span> </span>if</span> cond { </span>100</span>u32 </span>} </span>else </span>{ </span>12</span>u16 </span>} </span>} </span></code></pre> Both of these implement Debug</code>, and the syntax is valid this time around. So this should work right? Unfortunately not. We get the following error</a> if we try this:</p> error[E0308]: `if` and `else` have incompatible types </span> --> src/lib.rs:4:31 </span> \| </span>4 \| if cond { 100u32 } else { 12u16 } </span> \| ------ ^^^^^ expected `u32`, found `u16` </span> \| \| </span> \| expected because of this </span> \| </span>help: you could change the return type to be a boxed trait object </span> \| </span>3 \| fn foo(cond: bool) -> Box<dyn Debug> { </span> \| ~~~~~~~ + </span>help: if you change the return type to expect trait objects, box the returned expressions </span> \| </span>4 \| if cond { Box::new(100u32) } else { Box::new(12u16) } </span> \| +++++++++ + +++++++++ + </span></code></pre> Translating what the compiler is trying to tell us here: “You cannot use impl Debug</code> here, please use Box<dyn Debug></code> instead”. But taking the compiler up on their suggestion forces us to allocate the value on the heap instead of the stack, and to go through a pointer vtable to find the right value to dispatch to. But as we saw earlier: we could also just define an enum to get the same effect. The suggestion made by the compiler here is sensible, but not optimal</em>. Though the compiler is not at fault, there’s definitely something missing here. And I suspect if type-erased enums were supported by the language, the compiler would never have to suggest how to work around this in the first place, thus making the issue go away entirely.</p> Fortunately “type-erased enums” do</em> have a crate in the ecosystem which shows us what using this would feel like: auto_enums</a>. By adding just a few annotations our code suddenly works:</p> #[</span>auto_enum</span>(Debug)] </span>fn </span>foo</span>(</span>cond</span>: </span>bool</span>) -> impl Debug { </span> </span>if</span> cond { </span>100</span>u32 </span>} </span>else </span>{ </span>12</span>u16 </span>} </span>} </span></code></pre> auto_enums</code> uses a proc macro to generate an enum for each type in the code. We have to tell it which traits the output type should implement, but other than that it reads fairly naturally.</p> Looking at the code generate by the auto_enums</code> crate, I suspect that this should be fairly straight forward to implement in the compiler if we can get lang approval for it. Which at first glance doesn’t seem like a big language feature either. But I don’t know enough about this side of the language to appropriately judge what gotchas might apply (const, FFI, etc. all need to be considered). So I don’t know how difficult this might be in practice to see through to completion. But I’d be interested to hear from any lang/compiler folks whether my intuition here is right?</p> Comparing enums and structs</h2> Now that we’ve taken a look at both structs and enums, we can capture their capabilities in a table:</p> </th> Structs</th> Enums</th> Enums Fallback</th></tr></thead> Named</strong></td> struct Foo(.., ..)</code></td> enum Foo { .., .. }</code></td> -</td></tr> Anonymous</strong></td> (.., ..)</code></td> ❌</td> either</code> crate</td></tr> Type-Erased</strong></td> impl Trait</code></td> ❌</td> auto_enums</code> crate</td></tr> </tbody></table> As you can see, it's much easier to quickly create a struct than it is to create an enum. Fallbacks for quick enum creation exist in the ecosystem, but Even if both structs and enums can both hold rich data, the bar to using structs in Rust is lower than for enums. I would argue, significantly so.</p> Wrapping up</h2> We’ve shown the differences between struct and enum definitions. And shown how the lack of anonymous and type-erased enums is worked around today using crates from the ecosystem.</p> My current thinking is that fixing this asymmetry would be beneficial for Rust eventually. It’s a convenient piece of polish which could make daily driving Rust nicer. But it’s nothing fundamental that needs to be addressed straight away. So I don’t think it’s anything which requires being prioritized.</p> Anyway, I got thinking about this when chatting with folks about this earlier, and figured I’d put my thinking into words. In part to share the idea, and in part to learn how to publish things more quickly. If you liked this post, and would like to see what I make for dinner, follow me on Twitter</a>.</p> Appendix A: Examples</h2> Structs</h3> struct </span>MyStruct(</span>u32</span>, </span>u16</span>); </span> </span>// named type </span>fn </span>foo</span>() -> MyStruct { </span> MyStruct(</span>100</span>, </span>12</span>) </span>} </span> </span>// anonymous type </span>fn </span>foo</span>() -> (</span>u32</span>, </span>u16</span>) { </span> (</span>100</span>, </span>12</span>) </span>} </span> </span>// type erased type </span>fn </span>foo</span>() -> impl Debug { </span> MyStruct(</span>100</span>, </span>12</span>) </span>} </span></code></pre> Enums</h3> enum </span>MyEnum { </span> Left(</span>u32</span>), </span> Right(</span>u16</span>), </span>} </span> </span>// named type </span>fn </span>foo</span>(</span>cond</span>: </span>bool</span>) -> MyEnum { </span> </span>if</span> cond { MyEnum::Left(</span>100</span>) } </span>else </span>{ MyEnum::Right(</span>12</span>) } </span>} </span> </span>// anonymous type (currently unsupported) </span>fn </span>foo</span>(</span>cond</span>: </span>bool</span>) -> </span>u32 </span>\| </span>u16 </span>{ </span> </span>if</span> cond { </span>100</span>u32 </span>} </span>else </span>{ </span>12</span>u16 </span>} </span>} </span> </span>// type erased type (currently unsupported) </span>fn </span>foo</span>(</span>cond</span>: </span>bool</span>) -> impl Debug { </span> </span>if</span> cond { </span>100</span>u32 </span>} </span>else </span>{ </span>12</span>u16 </span>} </span>} </span></code></pre> Appendix B: Tables</h2> </th> Structs</th> Enums</th> Enums Fallback</th></tr></thead> Named</strong></td> struct Foo(.., ..)</code></td> enum Foo { .., .. }</code></td> -</td></tr> Anonymous</strong></td> (.., ..)</code></td> ❌</td> either</code> crate</td></tr> Type-Erased</strong></td> impl Trait</code></td> ❌</td> auto_enums</code> crate</td></tr> </tbody></table> Futures Concurrency III: select! Wed, 09 Feb 2022 00:00:00 +0000 In our last post on futures concurrency</a> we talked about the various ways futures in Rust can be awaited concurrently. But we didn't cover one mode: awaiting multiple futures concurrently and resolving them as soon as they're ready, without ever discarding any data. This mode of concurrency is what is conventionally provided by the futures::select!{}</code> macro</a>.</p> In this post we're going to take a look at how this mode of concurrency works, take a closer look at the issues select! {}</code> has, discuss Stream::merge</code> as an alternative, and finally we'll look at what the ergonomics might look like in the future. This post is part of the "Futures Concurrency"</em> series. You can find all library code mentioned in this post as part of the futures-concurrency</code></a> crate.</p> process concurrently, yield sequentially</h2> In our last post we wrote an example where we raced multiple futures, and got the output from the first future which resolves:</p> let</span> a = future::ready(</span>1</span>).</span>delay</span>(Duration::from_millis(</span>100</span>)); </span>let</span> b = future::ready(</span>2</span>).</span>delay</span>(Duration::from_millis(</span>200</span>)); </span>let</span> c = future::ready(</span>3</span>).</span>delay</span>(Duration::from_millis(</span>300</span>)); </span>assert_eq!([a, b, c].</span>race</span>().await, </span>1</span>); </span></code></pre> Using race</code> gets the future a</code> as soon as it resolves, but discards futures b</code> and c</code>, making their content unavailable. If we don't want to discard any futures we can use join</code>, but this requires waiting until all</em> futures have been resolved before we can operate on their output. What if we wanted something that sits between the two: How can we get the result of all futures, but process them one-by-one as soon as they've resolved?</p> If we pull up the table we introduced last post, we can now add a third column. Instead of waiting for all</em> outputs or just the first</em> output, we're now handling all outputs one-by-one, as soon as they're ready:</p> </th> Wait for all outputs</strong></th> Wait for first output</strong></th> Handle output one-by-one</strong></th></tr></thead> Continue on error</strong></td> Future::join</code></td> Future::try_race</code></td> ???</code></td></tr> Return early on error</strong></td> Future::try_join</code></td> Future::race</code></td> ???</code></td></tr> </tbody></table> This is the table we've been working with so far, and it makes sense to continue filling it out. But maybe if we disregard "fallibility" for a moment, there's another table we can create? Like race</code> we start handling items as soon as the first item is resolved, but instead of discarding the remaining items, like join</code> we are interested in all</em> items:</p> </th> Handle all items</strong></th> Discard some items</strong></th></tr></thead> Response starts on first item</strong></td> ???</td> Future::race</code></td></tr> Response starts on last item</strong></td> Future::join</code></td> ???</td></tr> </tbody></table> For the majority of this post we'll be focusing on the top-left of this new table: "response starts on first item" and "handle all items". We'll cover the bottom right mode of concurrency towards the end of this post.</p> Let's start by taking a look at how the futures</code> crate exposes this mode of concurrency first.</p> the futures::select! macro</h2> The futures</code> crate has a solution for this mode of concurrency through the select!{}</code> macro</a>. The tokio</code> runtime also has a variant of select! {}</code></a>, but it behaves slightly differently. Both the futures</code> and tokio</code> variants of select</code> behave largely the same, so the broader points we're making apply to both. However where relevant, we will look closer at how the two differ.</p> The select! {}</code> macro introduces a custom DSL</a> which resembles match</code> blocks, but with a few twists. Unlike a match</code> block which takes a single input, select! {}</code> takes multiple futures, creates named bindings to them when they resolve, and executes a block when it executes. It also has a complete</code> case (else</code> in the tokio</code> variant) which runs when no more data is available from any of the input futures.</p> Here's a basic example of how select!{}</code> is used, adapted from the async book</a>. This takes multiple asynchronous streams of numbers and sums them (playground</em></a>):</p> use </span>futures::stream::{</span>self</span>, StreamExt}; </span>use </span>futures::select; </span> </span>// Create two streams of numbers. Both streams require being `fuse`d. </span>let mut</span> a = stream::iter(vec![</span>1</span>, </span>2</span>, </span>3</span>]).</span>fuse</span>(); </span>let mut</span> b = stream::iter(vec![</span>4</span>, </span>5</span>, </span>6</span>]).</span>fuse</span>(); </span> </span>// Initialize the output counter. </span>let mut</span> total = </span>0</span>; </span> </span>// Process each item in the stream; </span>// break once there are no more items left to sum. </span>loop </span>{ </span> </span>let</span> item = select! { </span> item = a.</span>next</span>() => item, </span>// 1. Get the next future in the stream, </span> item = b.</span>next</span>() => item, </span>// assign its `Output` to `item` when it resolves. </span> complete => </span>break</span>, </span>// 2. The `complete` case runs when both streams are exhausted. </span> }; </span> </span>if let </span>Some(num) = item { </span> </span>// Increment the counter with the value from the stream. </span> total += num; </span> } </span>} </span> </span>// Validate the result. </span>assert_eq!(total, </span>21</span>); </span></code></pre> This example is simplified. It can roughly be broken into three steps:</p> Create the streams and fuse</code></a> them. fuse</code> makes it so once a Stream</code> returns None</code>, it guarantees to keep returning None</code> instead of potentially panicking 1</a></sup>.</li> Create the main processing loop using select!</code>, summing the numbers together.</li> Validate the result is correct.</li> </ol> ^{1</sup> Unfused streams are not guaranteed to panic if they are accessed again after they've been exhausted. But they definitely can</em> panic, and it's not unlikely that when we have generators they will, just like Future</code>, panic if they're called again after they've been exhausted.</p> </div>}Issues with futures::select!</h2> Now that we've talked about the select! {}</code> macro and the mode of concurrency it enables, it's time we start looking at the issues with it. Because oh boy are there many. My personal opinion has always been that select! {}</code> seems hard to use, and feels off. But in writing this post I've had to substantiate this feeling with actual evidence, and I don't like what I've learned. Not. At. All.</p> In this section we'll cover different aspects of how select! {}</code> falls short. But if you want my brief, summarized opinion: I view select! {}</code> as easy to misuse, hard to debug, difficult to learn, when used correctly</em> makes for incredibly hard-to-read code, and seems unlikely to being included in the stdlib or language.</strong> I think this qualifies as a strong statement</em>. Now let me back it up by sharing my reasoning for each part.</p> modifying "hello world" can lead to data loss</h3> select! {}</code> is not limited to implementing a single form of concurrency: it is flexible enough that it can be used to implement different concurrency primitives. This makes it hard for both the compiler and humans to validate whether it's used correctly. Which is not great when performing concurrent operations in a potentially multi-threaded system. This can be exceptionally hard to debug.</p> Let's take our earlier example, and instead of taking numbers from a stream and incrementing a counter, let's do something we might encounter in the real-world. For example, in one select arm we may want to process items from a stream, and in the other we may want to read data from some handle and send it into a channel:</p> // before </span>loop </span>{ </span> </span>let</span> item = select! { </span> item = a.</span>next</span>() => item, </span> item = b.</span>next</span>() => item, </span> complete => </span>break</span>, </span> }; </span> ... </span>} </span></code></pre> // after </span>loop </span>{ </span> futures::select! { </span> _ => </span>read_send</span>(&</span>mut</span> handle, &</span>mut</span> channel).</span>fuse</span>() => {} </span>// read data from a handle and send it to a channel </span> item => stream.</span>next</span>() => { ... } </span>// process data from a stream </span> complete => </span>break</span>, </span>// all work completed, stop the loop </span> } </span>} </span></code></pre> If you squint you can see that the shape of both loops remained roughly the same. The most significant change is that instead of calling stream.next()</code> in both arms, we have switched out one arm to use read_send</code> instead. We can no longer rely on the futures being fused, so we need to manually do that on the future now instead. But what you probably didn't expect is that this code is fundamentally broken, and your program will</em> lose data.</p> The second example is the exact pattern Tomaka covers in their 2021 post on async Rust</a>. They based this post on their experience writing production async Rust code for several years, and described this issue as follows (emphasis is Tomaka's):</p> It is always possible to solve this problem in some way, but what I would like to highlight is an even bigger problem: these kind of cancellation issues are hard to spot and debug</strong>. In the problematic example, all you will observe is that in some rare occasions some parts of the file seem to be skipped. There will not be any panic or any clear indication of where the problem could come from. This is the worst kind of bugs you can encounter.</strong></p> </blockquote> What's happening here is that we intended to express: "process concurrently, yield sequentially" semantics. But what we ended up writing was "race" semantics. The reason for this is that read_send</code> and stream.next()</code> are racing, and if stream.next()</code> completes first, the read_send</code> future is discarded. This can lead to data from handle</code> having being read, but not yet sent into the channel</code>. Which means the next time we read data from handle</code>, the cursor will have advanced, and data is now lost.</p> You may be wondering why stream.next()</code> does</em> work, but read_send</code> does not. The short explanation is that the state of the stream.next</code> future is entirely contained within stream</code>. Which means that if we re-create the future on each iteration, we can continue right where we left off. read_send</code> on the contrary has state local to the future. Which means when we recreate the future, we lose that state, regardless of how far we got.</p> In their post Tomaka appears to primarily attribute the issue they're experiencing to all futures being cancellable. And that makes sense: if future can't be cancelled, it can't lead to data loss. And if select!</code> only ever operates un cancellable futures, then we've successfully guarded against this issue! But in my opinion the issue is less with all futures being cancellable (I think this is good</a>), but instead that depending on how you configure select!</code> you may unintentionally be presented with "race" semantics</a>!</p> In general I'd recommend folks don't attempt to use select!</code> for this mode of concurrency and instead spawn (local) tasks instead. This is largely how over the years I've avoided to ever have to write select!</code> statements using futures. But because we're talking about the issues with select!</code> we should at least take a look at what a correct solution using select!</code> might look like:</p> // Create the future we're polling once, outside the loop so it </span>// doesn't drop between iterations. </span>let mut</span> read_send_fut = </span>read_send</span>(&</span>mut</span> handle, &</span>mut</span> channel); </span>loop </span>{ </span> futures::select! { </span> _ => read_send_fut => { </span> </span>// Whenever the last future resolves, we need to re-create it for the next loop. </span> read_send_fut = </span>read_send</span>(&</span>mut</span> handle, &</span>mut</span> channel); </span> } </span> item => stream.</span>next</span>() => { ... } </span> complete => </span>break</span>, </span> } </span>} </span></code></pre> Let me be honest: I don't feel good about this solution. We're instantiating the exact same future in two separate places. And because this is an example, we're lucky in this case that we only do this for the one future. In real-world examples we might want to repeat this more often. We could probably abstract the future instantiation using a function. But I'm hesitant to introduce further abstractions.</p> To me this code feels brittle, riddled with assumptions which are not directly obvious from the code, and it's becoming hard to see what it is that we're actually</em> trying to do here. And that's an issue. select!</code> is so flexible, it stops being able to accurately capture intent</em>. This makes it impossible for the compiler to validate we're using it right. And hard for people reading the code to understand what we're trying to do. Because it's possible to mis-use select!</code> it means that any diligent code-reviewer will manually need to validate select!</code> blocks are right, every time they're modified. Because the consequences of failing to do so can have very real consequences.</strong></p> An alternative to select! {}</code> should only expose a single concurrency mode from its API, making it impossible to accidentally opt-in to a different mode. One of Rust's core strengths is the ability for the compiler to validate our usage of APIs is correct. And for something as fundamental as a concurrency mode we should absolutely be making use of this.</p> halt points are hidden in a macro</h3> In my post on async cancellation</a>, I showed the following example displaying every point where code can be halted. You can think of "halting" as "this is where the function will potentially stop executing, so we need to think about how to clean up local state when it does":</p> // Regardless of where in the function we stop execution, destructors will be </span>// run and resources will be cleaned up. </span>async </span>fn </span>do_something</span>(</span>path</span>: PathBuf) -> io::Result<Output> { </span> </span>// 1. the future is not guaranteed to progress after instantiation </span> </span>let</span> file = fs::open(&path).await?; </span>// 2. `.await` and 3. `?` can cause the function to halt </span> </span>let</span> res = </span>parse</span>(file).await; </span>// 4. `.await` can cause the function to halt </span> res </span>// 5. execution has finished, return a value </span>} </span></code></pre> Something which makes select! {}</code> difficult to reason about with regards to halting (and cancellation) is that it obfuscates potential halt points by not requiring that they are marked with .await</code>:</p> loop </span>{ </span> </span>let</span> item = select! { </span> item = a.</span>next</span>() => item, </span>// 1. .await is hidden here, which can cause a halt </span> item = b.</span>next</span>() => item, </span>// 2. .await is hidden here, which can cause a halt </span> complete => </span>break</span>, </span> }; </span> ... </span>} </span></code></pre> Generally the rule is that halt points contained in function bodies are annotated using ?</code> or .await</code>. But select!</code> hides .await</code> statements internally, meaning they can no longer be seen in in the function body. Someone unfamiliar with select!</code>'s semantics may not expect it to cause the function to halt, which can lead to bugs. An alternative to select! {}</code> should use .await</code> to annotate it can halt.</p> fuse</code> requirements</h3> When a Future</code> is created through using the async</code> keyword, if it's polled after it's been completed, it will panic. Similarly, once we have generators in the language, creating a Stream</code> through an async gen fn</code> we would logically expect that to panic too if we call it after it's been completed 2</a></sup>.</p> ^{2</sup> The compiler guards against "poll-after-completion" errors in most cases because .await</code> takes ownership of futures. This means if you call .await</code> twice on the same future, it'll fail to compile with a "use of moved value" error. The "panic if this future is polled after it's completed" error is kind of like the runtime version of the move semantics guard, for when a future is manually polled to completion. While not required by the Future</code> trait, I think it's good practice for most futures to implement similar "poll-after-completion" behavior as futures created through the async</code> keyword.</p> </div> "fuse semantics" mean slightly different things for futures, iterators, and streams:</p>} Future</code></strong>: after a future has returned Poll::Ready</code> once, keep returning Poll::Pending</code> instead of panicking(ref</a>).</li> Iterator</code></strong>: after an iterator has returned None</code> once, keep returning None</code> instead of panicking (ref</a>).</li> Stream</code></strong>: after a stream has returned Poll::Ready(None)</code> once, keep returning Poll::Ready(None)</code> instead of panicking (ref</a>).</li> </ul> Futures passed to select!</code> must implement fuse semantics so that we can track the completion status of each individual future. If we poll a future and it This cannot be tracked inside the select! {}</code> macro because select! {}</code> does not take ownership of individual futures. Once a select! {}</code> block has yielded, the futures within it can be accessed again, which is often used to pass them back into another select! {}</code> macro on a subsequent loop.</p> "Does my future implement fuse semantics" is checked during compilation in the futures::select!</code> macro</a>, but left as an exercise to the user when using tokio::select!</code></a>. This makes the futures</code> variant more resilient than the tokio</code> variant, but in both cases it's annoying that we have to care about this at all because Future</code> and async/.await</code> were not designed to default to implement fuse semantics.</p> This mismatch causes select! {}</code> requires users to need to educate themselves on what fusing futures is, how to do it, and how not to be caught by runtime errors. select! {}</code> as an abstraction does not fit well with the rest of async Rust, and as a result users end up needing to learn how to work around this mismatch. Which does not make for a great experience, and would be great if we could evade front-loading when teaching folks about async Rust.</p> trait signatures</h3> select! {}</code> can not only operate on futures, it can operate on asynchronous streams too. There is no real counterpart to Future</code> in the stdlib (the syn version of "asynchronous value" is just "a value"), but the counterpart to AsyncIterator</code> (née: Stream</code>) is Iterator</code>. An unfortunate consequence of select! {}</code> is that it has lead to a mismatch between the signatures of futures::FusedStream</code></a> and FusedIterator</code></a>. FusedStream</code> has a mandatory method: is_terminated</code> which needs to be implemented. FusedIterator</code> does not have this method.</p> The is_terminated</code> method on Future</code> allows a third state to be represented in the Future</code> poll model: "this future has completed". For Stream</code> we can represent this state by yielding Poll::Ready(None)</code>. But in Future</code> we only have the choice between Poll::Ready(T)</code> and Poll::Pending</code>, neither of which means "I'm done". As remarked in the original issue discussing FusedFuture</code></a>, the need for having the is_terminated</code> method would go away if select! {}</code> would have an internal loop</code> statement. If the loop lives internally, state tracking can be moved interally as well, and we no longer have the same issues. But unfortunately select_loop!</code> was deemed as being too inflexible</a>, and instead the choice was made to go with FusedFuture</code> / FusedStream</code> instead 3</a></sup>.</p> ^{3</sup> select_loop!</code> is so close</em> to what we're proposing as the solution in this post. I only found out about it towards the end of editing this post, but it feels like this was just a few iterations away from what we're proposing we do here instead.</p> </div>}The reason why discussing trait mismatches is important is because a guiding principle for of the async foundations WG is parity between sync and async Rust</a>:</p> Async Rust should be a small delta atop Sync Rust. People who are familiar with sync Rust should be able to leverage what they know to make adopting Async Rust straightforward. Porting a sync code base to async should be relatively smooth: just add async/await, adopt the async variants of the various libraries, and you're done.</p> </blockquote> Introducing FusedFuture</code>, FusedStream</code> and select! {}</code> into the stdlib would mean we'd need to take one of three decisions:</p> We accept that FusedStream</code> and FusedIterator</code> have a different signature, and all the consequences that carries regarding parity with std rust</a>.</li> We re-work select! {}</code> to not rely on FusedFuture::is_terminated</code>. I believe this has been tried though, and it didn't work out.</li> We re-work select! {}</code> to not require fuse semantics. This would require select! {}</code> take ownership of futures, which would lead to data loss.</li> </ol> select! {}</code>'s relationship to the Fused</code> traits is not good, and all options how to resolve it carry distinct downsides. The best solution would be to look for alternatives to select! {}</code> which implement the same concurrency mode, but do not rely on fuse semantics on the types they operate on.</p> debugging errors is hard</h3> If you misuse select! {}</code>, how do you find out? Implementing race</code> semantics where you didn't mean will lead to data occasionally</em> being discarded. Depending on the data you're reading this can lead to hard-to-detect bugs. There's no guarantees errors will be thrown, it's not obvious from the code that we're racing, and we have few (if any) tools or lints to detect races like these. The best chance at catching issues with this before they occur is to have a thorough test suite which can pinpoint this error before it's shipped. But in my experience this is not a given.</p> Debugging work is often 90% understanding the bug, and 10% fixing the bug. And in order to catch bugs with select! {}</code> you need to understand how the macro works, how async cancellation works in Rust, find out exactly where it's used, and finally: employ a fix. This is the type of work that even experts in async Rust might struggle with (hi).</p> It doesn't mean that we can't do anything about this though. We could apply runtime probing to find out where futures are being cancelled in a loop. Or static lints which catch potential mis-configurations of select! {}</code>. But that's speculation, and will never be as good a fix as having APIs where bad states are unrepresentable.</p> select!</code> and the stdlib</h3> Objectively select! {}</code> has many other issues, the first being its relation to the stdlib. As we're looking to add more async functionality to the stdlib</a>, foundational components for async concurrency will be part of this. So it's worth asking how we could add select! {}</code>'s functionality to the stdlib.</p> Introducing select! {}</code> as a macro would be the first option. But if we introduced it today, it would immediately become one of the most complicated macros in the stdlib. select! {}</code> really is shaped like a control-flow primitive, and Rust tends to prefer keywords over macros for control flow 4</a></sup>. Rust cares deeply about diagnostics, and for something as fundamental as control flow we would want to provide comprehensive errors, explainers, and suggestions.</p> ^{4</sup> The try!</code> macro became ?</code>. The await!</code> macro became .await</code>. Control flow constructs in Rust tend to be introduced as keywords instead, but the bar for those is incredibly high.</p> </div>}Adding select</code> as a keyword to Rust would possibly be harder; the barrier for keywords in Rust is incredibly high. In order for a new keyword to be introduced, an irrefutable case must be presented. For .await</code> we proved that it enabled things that could otherwise only be done using unsafe</code>. const</code> in generic position were added because it enabled computation within the type system. select</code> in comparison merely adds a fifth mode of concurrency to async Rust 5</a></sup>. This is hardly comparable in impact. Especially if we can prove that we can provide the same concurrency primitives entirely using library functions.</p> ^5</sup>crossbeam_channel::select</code></a> exists for non-async Rust and is similar in shape. But it operates over channels only, and cannot yet be mixed with async Rust. Any proposal for a select</code> keyword would need to carefully consider how select</code> would interact with both async and non-async Rust.</p> </div> concurrent stream processing with Stream::merge</code></h2> So far we've taken a look at the mode of concurrency we're trying to implement, and the issues the select! {}</code> macro has. This is the part where we move past that and show a surprisingly simple alternative solution to this type of concurrency: Stream::merge</code>. We'll be stepping away from futures for a second to talk about streams, but we'll get back how this relates to futures as well in a bit.</p> Fundamentally what we want to be doing is awaiting multiple streams concurrently; processing items from either stream as soon as they're ready. In async-std</code> we introduced Stream::merge</code> about two years ago (!) for this exact purpose. It now also exists in futures-concurrency</code>, the companion library to this series. But for this example we'll stick to async-std</code>.</p> The way Stream::merge</code> works is that it takes two streams, and merges</em> them as if they were a single stream. Data comes out as soon as it's ready, and can be handled just like any other stream:</p> use </span>async_std::prelude::; </span>use </span>async_std::stream; </span> </span>// Declare our streams. </span>let</span> a = stream::iter(vec![</span>1</span>, </span>2</span>, </span>3</span>]); </span>let</span> b = stream::iter(vec![</span>4</span>, </span>5</span>, </span>6</span>]); </span> </span>// Initialize the output counter. </span>let mut</span> total = </span>0</span>; </span> </span>// Combine both streams, and add each value to the total. </span>a.</span>merge</span>(b).</span>for_each</span>(\|</span>num</span>\| total += num).await; </span> </span>// Validate the result. </span>assert_eq!(total, </span>21</span>); </span></code></pre> This code is functionally equivalent to the select! {}</code> loop we introduced at the start of this post. Except it's significantly</em> less code, does not require manually fuse</code>ing streams (Merge</code> tracks the state), and importantly: does not introduce any new syntax [^gripe].</p> This is not even the shortest version this can be, using the stream::Sum</code></a> trait we can shorten this specific example to:</p> use </span>async_std::prelude::; </span>use </span>async_std::stream; </span> </span>let</span> a = stream::from_iter(vec![</span>1</span>, </span>2</span>, </span>3</span>]); </span>let</span> b = stream::from_iter(vec![</span>4</span>, </span>5</span>, </span>6</span>]); </span>assert_eq!(a.</span>merge</span>(b).</span>sum</span>().await, </span>21</span>); </span></code></pre> Conceptually this seems so simple, it almost feels too good to be true. But it isn't: there are no hidden Unpin</code> bounds, no secret FusedStream</code> requirements, or unexpected gotchas. We got to this solution by closely analyzing the mode of concurrency select! {}</code> exposes, and finding prior art for it in rxjs</a>.</p> Concurrently processing a mix of futures and streams</h2> Perhaps you're wondering whether Stream::merge</code> can only replace select! {}</code> loops in simple cases. What about differently-typed streams with different signatures and individual futures mixed in. That's exactly what select! {}</code> was designed for. For the sake of brevity I'll skip the select! {}</code> variant and only show how well Stream::merge</code> is capable of handling this6</a></sup>:</p> ^{6</sup> This reminds me we made mistakes in async-std</code>. stream::once</code> really should be able to take a Future</code> rather than a regular value. This example pretends we didn't make that mistake.</p> </div>}use </span>async_std::prelude::; </span>use </span>async_std::stream; </span> </span>// Create a shared output enum, used by all stream. </span>enum </span>Message { </span> Num(</span>u8</span>), </span> Text(&</span>'static str</span>), </span>} </span> </span>// Create a variety of streams from futures and iterators, and map them to the </span>// same output type </span>let</span> a = stream::from_iter([</span>1</span>, </span>2</span>, </span>3</span>]).</span>map</span>(Message::Num); </span>let</span> b = stream::from_iter(["</span>chashu</span>", "</span>nori</span>"]).</span>map</span>(Message::Text); </span>let</span> c = stream::once(async { "</span>hello</span>" }).</span>map</span>(Message::Text); </span> </span>// Merge all streams and handle each output one-by-one. </span>let mut</span> s = a.</span>merge</span>(b).</span>merge</span>(c); </span>while let </span>Some(msg) = s.</span>next</span>().await { </span> </span>match</span> msg { </span> Num(n) => println!("</span>received a number: </span>{}</span>", n), </span> Text(s) => println!("</span>received a string: </span>{}</span>", s), </span> } </span>} </span></code></pre> We can even add short-circuiting by using a generalized abort mechanism like stop-token</code></a>, or adding another enum variant that maps to break</code> whenever called.</p> This shows how Stream::merge</code> implements the "process items as soon as they're available" mode of concurrency, the same as select! {}</code>. And importantly, it does so using regular Rust types and techniques.</p> The least comfortable part of what we've written is type unification using enum Message {}</code> and map(Message::Type)</code>. But making sure types align is a common Rust issue, and using an enum for that is the common solution. The semantics here are a lot less special</em> than those introduced by select! {}</code>, which should make this significantly easier to get the hang of. Especially if we document it right.</p> If we're thinking about a future where the stdlib provides a complete</em> solution for futures concurrency out of the box, then the options for select! {}</code> look rather unappealing. We could choose to ship the macro as-is, and forever deal with manual fuse</code> calls, diverging traits [1</a>, 2</a>], and daunting errors.</p> Or we could promote select</code> into a keyword. This could do away with the requirement to manually invoke fuse</code> for us. But would come with the downside that we now have a syntax which is unlike anything else. The arms are different from those found in match</code>, and it would likely be uniquely scoped to async contexts. Introducing new keywords is an incredibly</em> high barrier to clear, and have my doubts whether select</code> could make it.</p> Concurrently processing futures in a loop</h2> Earlier in this post we looked at Tomaka's post which featured data-loss issues with select! {}</code></a>. Let's bring up the example verbatim from the post:</p> // open our IO types </span>let mut</span> file = ...; </span>let mut</span> channel = ...; </span> </span>// This loop re-creates the `read_send` future on each iteration. That means if </span>// `socket.read_packet` completes, `read_send` may have read data from the file, </span>// but not finished writing it to a channel, which can lead to data loss </span>loop </span>{ </span> futures::select! { </span> _ => </span>read_send</span>(&</span>mut</span> file, &</span>mut</span> channel) => {}, </span>// uh oh, data loss can happen here </span> some_data => socket.</span>read_packet</span>() => { </span> </span>// ... </span> } </span> } </span>} </span></code></pre> The issue was that we accidentally opted into "race" semantics when we didn't mean to, which could lead to data loss. In their post, Tomaka mentions there are different ways of solving this, including my personal choice: spawn each individual operation on a separate task and await them</a> But for the sake of example, let's take a look at how we can implement the logic above using Stream::merge</code>:</p> // open our IO types </span>let mut</span> file = ...; </span>let mut</span> channel = ...; </span> </span>// Create our shared message type </span>enum </span>Message { </span> None, </span> Data<Vec<</span>u8</span>>>, </span>} </span> </span>// Create two streams by creating and awaiting futures in a loop. We unify the output </span>// of both futures through the `Message` enum. </span>let</span> a = stream::repeat_with(\|\| async { </span>read_send</span>(&</span>mut</span> file, &</span>mut</span> channel).await }).</span>map</span>(Message::None); </span>let</span> b = stream::repeat_with(\|\| async { socket.read_packet.await }).</span>map</span>(Message::Data); </span> </span>// Merge both the streams and wait for their output one-by-one. </span>let mut</span> s = a.</span>merge</span>(b); </span>while let </span>Some(msg) = s.</span>next</span>().await { </span> </span>match</span> msg { </span> Message::None => </span>continue</span>, </span> Message::Data(s) => ..., </span> } </span>} </span></code></pre> This creates an endlessly looping stream for sending data from the file into the channel, and another endlessly looping stream to read data from the socket. It's definitely more verbose than the original select! {}</code> example, but we also have 100% less data loss. Instead of dropping and re-creating read_send</code> each time read_packet</code> makes progress, both just keep moving forward the way you would expect them to.</p> Would I write this for a project I'm working on? Probably not; I'd likely use async-task-group</code></a> instead. But I think this is helpful to show a practical example of a well-publicized issue with select! {}</code> and how Stream::merge</code> would help guard against that too.</p> Fairness</h2> There is one issue with merge</code> we haven't covered yet: fairness. Let's look at the example which merges three streams into one:</p> let</span> a = stream::repeat(</span>1</span>).</span>take</span>(</span>3</span>); </span>let</span> b = stream::repeat(</span>2</span>).</span>take</span>(</span>3</span>); </span>let</span> c = stream::repeat(</span>3</span>).</span>take</span>(</span>3</span>); </span>let</span> d = stream::repeat(</span>4</span>).</span>take</span>(</span>3</span>); </span> </span>let</span> s = a.</span>merge</span>(b).</span>merge</span>(c).</span>merge</span>(d); </span></code></pre> Streams a</code>, b</code>, and c</code> will yield a number exactly three times. Because they always have an item available futures will always immediately yield the data that they have, instead of breifly waiting. Which means that if we implement our merge</code> function without accounting for this, we would end up with:</p> assert_eq!(s.</span>collect</span>(), vec![</span>1</span>, </span>1</span>, </span>1</span>, </span>2</span>, </span>2</span>, </span>2</span>, </span>3</span>, </span>3</span>, </span>3</span>, </span>4</span>, </span>4</span>, </span>4</span>]); </span></code></pre> This is not what we want: this has exactly the same semantics of Stream::chain</code></a>, and will exhaust streams in-order. Ideally we'd like a random mix of 1</code>s, 2</code>s, 3</code>s, and 4</code>s. In order to do that we need to randomly pick between the items available from merge.</code> Unfortunately because we're merging streams by chaining calls to merge</code> we can only pick between two streams at a time 7</a></sup>. This causes the chance we'll pick an item from a stream to be skewed. For the first 3 items in the stream, what we want is for each number to have exactly a 25%</code> chance to be picked. But instead the chance we'll get a number is skewed as follows:</p> number 1</code>: 50% chance</li> number 2</code>: 25% chance</li> number 3</code>: 12.5% chance</li> number 4</code>: 12.5% chance</li> </ul> The way to fix this is by making merge</code> aware of all</em> streams we're merging, so we can pick fairly between them. We could do this either by creating a merge!</code> macro, or by doing what we chose to do in futures-concurrency</code></a>: implement a trait for tuples [^merge-macro]. Here's what using that looks like today:</p> ^{7</sup> The reason for this is that when we call a.merge(b)</code> the merge</code> function only knows about a</code> and b</code>. If b</code> holds the stream produced by c.merge(d)</code>, then the probabilities divided between a</code>, b</code>, c</code> are now skewed. This is really annoying, and the only way we can resolve this is if we know how many items we need to choose from ahead of time</em>.</p> </div>}let</span> a = stream::repeat(</span>1</span>).</span>take</span>(</span>3</span>); </span>let</span> b = stream::repeat(</span>2</span>).</span>take</span>(</span>3</span>); </span>let</span> c = stream::repeat(</span>3</span>).</span>take</span>(</span>3</span>); </span>let</span> d = stream::repeat(</span>4</span>).</span>take</span>(</span>3</span>); </span> </span>let</span> s = (a, b, c, d).</span>merge</span>(); </span></code></pre> This will gives us exactly the probability we're looking for, ensuring one stream will not consistently be exhausted before another:</p> number 1</code>: 25% chance</li> number 2</code>: 25% chance</li> number 3</code>: 25% chance</li> number 4</code>: 25% chance</li> </ul> In past post's we've discussed the arity of the join</code> APIs</a> and its API consequences at length. But we haven't done so considering fairness</em> specifically. For the join</code> family of APIs, it's generally an all-or-nothing approach, so fairness is as not much of an issue 8</a></sup>. But for race</code>, which only returns a single item, it might be. And so the same arguments we're making here in favor of a trait-based approach for merge</code> apply to race</code> as well.</p> ^{8</sup> Future::try_join</code> can return a single item when handling an error, so fairness could</em> be an issue. So it's not an exact science. But I'm not sure how important that is though. Maybe we just should account for it?</p> </div>}Shiny Future</h2> Thinking about a future where we introduce Stream::merge</code> (or more likely</a>: AsyncIterator::merge</code>) into the stdlib paints a wholly different picture. It already works well as-is. But as we add more features to the language that we want anyway, using Stream::merge</code> would in turn become nicer to use as well. So let's have fun shall we. What if Rust had all</em> the language features, what could our code look like then?:</p> async iteration so we can write async for</code> loops</li> match</code> shorthands so we can reduce some of the nesting</li> Merge</code> implemented directly on tuples.</li> an IntoAsyncIterator</code> trait implemented in all the same places as IntoIterator</code> is</li> std::stream</code> renamed to std::async_iter</code></li> </ul> use </span>std::async_iter; </span> </span>// The shared output type </span>enum </span>Message { </span> Num(</span>u8</span>), </span> Text(&</span>'static str</span>), </span>} </span> </span>// Create our streams and map them to the shared output type. </span>let</span> a = [</span>1</span>, </span>2</span>, </span>3</span>].</span>into_async_iter</span>().</span>map</span>(Message::Num); </span>let</span> b = ["</span>chashu</span>", "</span>nori</span>", "</span>lily</span>"].</span>into_async_iter</span>().</span>map</span>(Message::Text); </span>let</span> c = async_iter::once(async { "</span>hello</span>" }).</span>map</span>(Message::Text); </span> </span>// Merge the streams, iterate over them, and handle the output sequentially. </span>for</span> await msg in (a, b, c).</span>merge</span>() </span>match </span>{ </span> Num(n) => println!("</span>received a number: </span>{n}</span>"), </span> Text(s) => println!("</span>received a string: </span>{s}</span>"), </span>} </span></code></pre> So far these features seem somewhat plausible: these are all features that are fairly straight-forward syntactic sugar for existing patterns. And importantly: none of these features are specific to the operations we're doing here. Which means that even if we're looking at possible new language features here, none of these are specific to async concurrency. But what if we really</em> took it further. Let's dream big for a moment here. What if Rust also had:</p> "async overloading"</a> and an async overload for IntoIter</code></li> async IntoIter</code> implemented for all T: Future</code></li> anonymous enums</a></li> type ascription in match statements</li> </ul> // Create our streams and map them to the shared output type. </span>// `into_iter` here assumes "async overloading". </span>let</span> a = [</span>1</span>, </span>2</span>, </span>3</span>]; </span>let</span> b = ["</span>chashu</span>", "</span>nori</span>", "</span>lily</span>"]; </span>let</span> c = async { "</span>hello</span>" }; </span> </span>// Merge the streams, iterate over them, and handle the output sequentially. </span>// `merge` expects `T: async IntoIter`, which all types implement, so it just works. </span>// The type of `msg` here is `u8 \| &str`. This is inferred from the input types </span>// and represented in the match statement. </span>for</span> await msg in (a, b, c).</span>merge</span>() </span>match </span>{ </span> n: </span>u8 </span>=> println!("</span>received a number: </span>{n}</span>"), </span> s: &</span>str </span>=> println!("</span>received a string: </span>{s}</span>"), </span>} </span></code></pre> I can't say for sure whether this is a good idea or not. We're actively investigating if async overloading can work. And I have no clue if an async Iterator for T where T: Future</code> blanket impl is a good idea. But the point of this example is to have fun with it. To go big on the "what if". Maybe none of these features will ever land. But it's still a useful exercise to understand to which degree ergonomics are inherent of the API, and to which degree they're tied to other factors.</p> The future of Rust is what we make of it, so who knows if any of this will ever happen. But even if it doesn't: we've already shown that achieving this is possible using today's Rust. What we're showing here is purely just to make it so we need to write less code to achieve the same effect.</p> Addendum: ergonomics explorations for operating exclusively on futures</h2> Hi, editor Yosh here. I'm adding this section close to the time I'd hit publish on the post. My co-worker nrc</a> helped review this post, and they asked me whether I could show an example using futures exclusively.</p> We could directly implement merge</code> on Future</code>, creating a Stream</code>, but that wouldn't lend itself well to chaining multiple futures: a.merge(b).merge(c)</code> first creates a stream from a.merge(b)</code>, then tries to merge future c</code> to it, which won't work.</p> Working around this problem requires implementing the Merge</code> trait for Future</code> containers as well, which would not work on stable Rust today. We'd want to use negative trait bounds to express the right bounds:</p> impl [T; N] where N: IntoFuture + !IntoStream</code></li> impl [T; N] where N: IntoStream + !IntoFuture</code></li> </ul> And even with these bounds, merging Stream</code> and Future</code> types would still</em> mean we'd require casting all futures to streams first. We could overcome this by having a third trait, IntoMergable</code> or something, and then making our generic container impls rely on that:</p> impl IntoMergable for T where T: IntoFuture + !IntoStream</code></li> impl IntoMergable for T where T: IntoStream + !IntoFuture</code></li> [T; N] where N: IntoMergable</code></li> </ul> But that doesn't seem great. In general I think it's worth assessing the following considering ergonomics in this order:</p> How often does merge</code>-ing futures come up in practice?</li> Is that often enough that it would warrant changes to the language?</li> Would impl IntoStream for T: IntoFuture</code> be a feasible?</li> </ol> As I mentioned in earlier examples, my intuition is that whenever we want to merge</code> futures, we likely could solve the problem more elegantly using task::spawn{,_local}</code>. Meaning if we design APIs such as TaskGroup</code></a> well, we could carry Stream::merge</code>-like semantics through this. In fact, that is exactly what Swift's TaskGroup</code> type</a> does. It can asynchronously iterate out all results from the group, allowing the caller to handle them one-by-one.</p> So to summarize this side-adventure: if we're looking to improve the semantics of using Stream::merge</code> using futures only, there are a fair number of questions and avenues we should explore before we draw any definitive conclusions.</p> The async Rust concurrency tables: completed</h2> We can now begin filling in the original async Rust concurrency table using Stream::merge</code>. Because Stream::merge</code> enables on items to be operated one at the time, it can both "continue on error" and "return early on error":</p> let</span> s = a.</span>merge</span>(b); </span> </span>// continue on error </span>while let </span>Some(item) in s { </span> </span>// do something with `item`. </span>} </span> </span>// return early on error </span>while let </span>Some(item) in s { </span> </span>let</span> item = item?; </span> </span>// do something with `item`. </span>} </span></code></pre> This means Stream::merge</code> allows you to choose from either short-circuiting behavior:</p> </th> Wait for all outputs</strong></th> Wait for first output</strong></th> Handle output one-by-one</strong></th></tr></thead> Continue on error</strong></td> Future::join</code></td> Future::try_race</code></td> Stream::merge</code></td></tr> Return early on error</strong></td> Future::try_join</code></td> Future::race</code></td> Stream::merge</code></td></tr> </tbody></table> Regarding our second table, Stream::merge</code> enables the concurrency mode where we "handle all items" and "start responding on the first item". This is the top-left of the table. But what about the case where we're only interested in the last item, and want to discard the rest? It turns out that that's a subset of the functionality exposed by Stream::merge</code>, which we can get by using Stream::last</code></a>:</p> </th> Handle all items</strong></th> Discard some items</strong></th></tr></thead> Response starts on first item</strong></td> Stream::merge</code></td> Future::race</code></td></tr> Response starts on last item</strong></td> Future::join</code></td> Stream::merge</code> + Stream::last</code></td></tr> </tbody></table> Seeing this table as it is, you might wonder what the difference is between e.g. Future::join</code> and Stream::collect</code>. Or converting futures into streams and then breaking on the first item. You'd be right: in theory we could model everything using streams and call it a day. But in practice it wouldn't feel good.</p> The reason why Stream::merge</code> is the right answer for combining N futures is because it allows us to express the time aspect of the operation</strong>. We don't want all or nothing. We want items, in any order, as soon as they're made available to us. Streams are "data over time". And even if the data is originally held in futures, the time aspect can only be expressed using streams.</strong></p> In contrast, the Future::join</code> and Future::race</code> family of operations do not rely on the same time aspect. join</code> yields all items. race</code> just yields the one. So converting futures via streams back to singular items is an indirect mapping which ends up feeling clunky to author. And worse: it creates opportunities to introduce bugs. Future::join</code> and Future::race</code> may be subsets of Stream::merge</code>, but they are unambiguous in what they express which makes them a useful abstraction.</p> Overall this We might still want to change the names of the concurrency combinators later on. But the core functionality, shape, and usage all feels stable and consistent.</p> Conclusion</h2> In this post we've looked at Rust's 5th mode of concurrency: "Await multiple futures concurrently and operate on them as soon as they're ready." And in passing also covered the 6th mode of concurrency: "await multiple futures concurrently and discard all values but the last". We've shown how this can be handled using the futures::select!</code> macro. How this can better be handled using Stream::merge</code>. And talked about the various considerations and future directions we have for this design.</p> We've also done a real-world case analysis of an accidental mis-use of the select!</code> macro, and shown how Stream::merge</code> would've prevented the same issue from happening by making the failure case unrepresentable. Overall Stream::merge</code> presents itself as a viable alternative to select!</code> for the "handle one future at a time without ever discarding any" model of concurrency.</p> Overall I'm incredibly pleased with the progress we've made with this series. After nearly three years of work we finally have documented all modes of concurrency for async Rust, and their corresponding interfaces. We can now start experimenting with naming, polish, documentation, and then possibly (finally!) start bringing some of these things into mainline Rust via RFCs. Things will likely change between now and then, but it's good to finally stop needing to think about semantics, and being able to focus more on ergonomics. It's been a slow process, but I'm incredibly happy with the outcome. And thank you for following along over all of these years!</p> If you liked this post and would like to see my cats, you can follow me on Twitter</a>.</p> Thanks to: Ryan Levick</a>, Nick Cameron</a> and Irina Shestak</a> for reviewing drafts of this post.</em></p> Appendix: Async Rust Concurrency Operations</h2> These are all the concurrency operations we want to be able to express in async Rust.</p> 1. Concurrency adapters by fallibility mode and output handling</h4> </th> Wait for all outputs</strong></th> Wait for first output</strong></th> Handle output one-by-one</strong></th></tr></thead> Continue on error</strong></td> Future::join</code></td> Future::try_race</code></td> Stream::merge</code></td></tr> Return early on error</strong></td> Future::try_join</code></td> Future::race</code></td> Stream::merge</code></td></tr> </tbody></table> 2. Concurrency adapters by response start and output handling</h4> </th> Handle all items</strong></th> Discard some items</strong></th></tr></thead> Response starts on first item</strong></td> Stream::merge</code></td> Future::race</code></td></tr> Response starts on last item</strong></td> Future::join</code></td> Stream::merge</code> + Stream::last</code></td></tr> </tbody></table> Uninit Read/Write Tue, 07 Dec 2021 00:00:00 +0000 The AsyncRead</code> and AsyncWrite</code> traits are async versions of the Read</code> and Write</code> traits in Rust. They're core to async Rust, providing the interface to read and write bytes from for example the filesystem and network. But both the async and non-async variants have an open issue: how can we use these traits to write data into uninitialized memory?</p> In this post we look at the problem of reading into unitialized memory using Read</code>/Write</code>, and at possible solution we could introduce to solve this.</p> Showing the problem</h2> Both async and non-async Rust share the same issue for both Read</code> and Write</code> traits. So let's take use the non-async Read</code> trait as our example. It's defined as follows:</p> pub trait </span>Read { </span> </span>fn </span>read</span>(&</span>mut </span>self</span>, </span>buf</span>: &</span>mut</span> [</span>u8</span>]) -> Result<</span>usize</span>>; </span>} </span></code></pre> Calling Read::read</code> will read bytes into a mutable slice of memory, and return an io::Result</code> containing either the number of bytes read, or an error. Usage typically is something like this:</p> // open a file and init the buffer </span>let mut</span> f = File::open("</span>foo.txt</span>")?; </span>let mut</span> buffer = vec![</span>0</span>; </span>1024</span>]; </span> </span>// read up to 1024 bytes </span>let</span> read = f.</span>read</span>(&</span>mut</span> buffer)?; </span>let</span> data = buffer[..read]; </span></code></pre> This will read 1024 bytes of the file into the buffer. But it comes with a slight inefficiency: we're writing zeroes into the buffer, and then immediately writing more data into the buffer after that. This means we're doing a double-write, which is not as efficient as it could be</strong> (report</a>). Ideally we'd be able to do the following:</p> // open a file and init the buffer </span>let mut</span> f = File::open("</span>foo.txt</span>")?; </span>let mut</span> buf = Vec::with_capacity(</span>1024</span>); </span>// reserve capacity, but don't initialize </span> </span>// read up to 1024 bytes </span>let</span> read = f.</span>read</span>(&</span>mut</span> buf)?; </span>let</span> data = buffer[..read]; </span></code></pre> But this doesn't work. Even if the capacity</em> is 1024, because we haven't initialized the vector it's as if we passed an empty slice, and the following assertion will always hold:</p> assert_eq!(data.</span>len</span>(), </span>0</span>); </span></code></pre> And this is the problem this post is about: our slices can't have uninitialized memory, and our vectors which can</em> have uninitialized memory are always dereferenced into slices first. In order to support writing into uninitialized memory over Read</code> bounds we need to resolve this.</p> Solution 1: unitialized slices</h2> The current thinking is that we can solve this issue by introducing "slices of unitialized memory" and extending the IO traits with methods dedicated to taking these slices. This was proposed and subsequently merged as part of RFC 2930: read_buf</a> (tracking issue</a>).</p> The RFC is worth a read if you want a closer look at the problem space. But the gist of the solution it proposes is as follows:</p> Add a new ReadBuf</code> type which wraps [MaybeUninit::<u8>::uninit(); N];</code></li> Add an optional read_buf</code> method which takes &mut ReadBuf<'_></code> instead of &mut [u8]</code>.</li> Users who want to read into uninitialized memory can implement and use read_buf</code>.</li> </ul> The RFC provides the following usage example:</p> let mut</span> buf = [MaybeUninit::<</span>u8</span>>::uninit(); </span>8192</span>]; </span>let mut</span> buf = ReadBuf::uninit(&</span>mut</span> buf); </span> </span>loop </span>{ </span> some_reader.</span>read_buf</span>(&</span>mut</span> buf)?; </span> </span>if</span> buf.</span>filled</span>().</span>is_empty</span>() { </span> </span>break</span>; </span> } </span> </span>process_data</span>(buf.</span>filled</span>()); </span> buf.</span>clear</span>(); </span>} </span></code></pre> With the following definition for Read::read_buf</code> provided by default, intended to be modified by implementers:</p> impl </span>Read </span>for </span>MyReader { </span> </span>fn </span>read_buf</span>(&</span>mut </span>self</span>, </span>buf</span>: &</span>mut </span>ReadBuf<'_>) -> io::Result<()> { </span> </span>let</span> n = </span>self</span>.</span>read</span>(buf.</span>initialize_unfilled</span>())?; </span> buf.</span>add_filled</span>(n); </span> Ok(()) </span> } </span>} </span></code></pre> This achieves the goal it sets out to do: we successfully can read data into uninitialized memory. However for users it requires a little more work to setup:</p> // Read into uninit memory. </span>let mut</span> buf = [MaybeUninit::<</span>u8</span>>::uninit(); </span>1024</span>]; </span>let mut</span> buf = ReadBuf::uninit(&</span>mut</span> buf); </span>file.</span>read_buf</span>(&</span>mut</span> buf)?; </span>let</span> data = buf.</span>filled</span>(); </span> </span>// Read into init memory </span>let mut</span> buf = [</span>0</span>; </span>1024</span>]; </span>let</span> read = file.</span>read</span>(&</span>mut</span> buf)?; </span>let</span> data = buf[..read]; </span></code></pre> Overall this is not too bad, and gets us in the right direction. In order to write bytes to the heap instead of the stack I believe you can wrap the buf in a Box::new</code> 1</a></sup> 2</a></sup>. The main downsides are that it requires using a dedicated type just for this purpose, and it's slightly more verbose.</p> ^1</sup>I believe with placement new (box</code> keyword</a>) this might save yet another copy.</p> </div> ^2</sup>Or as maybewaffle pointed out</a>: this could be used with Vec::spare_capacity_mut</code> as well.</p> </div> Solution 2: uninitialized vectors</h2> Since August 2020, the Vec::spare_capacity_mut</code></a> method has been available on nightly. This returns the remaining spare capacity of the vector as a slice of MaybeUninit<T></code>. The docs show the following usage example:</p> let mut</span> v = Vec::with_capacity(</span>10</span>); </span>let</span> uninit = v.</span>spare_capacity_mut</span>(); </span> </span>uninit[</span>0</span>].</span>write</span>(</span>0</span>); </span>uninit[</span>1</span>].</span>write</span>(</span>1</span>); </span>uninit[</span>2</span>].</span>write</span>(</span>2</span>); </span> </span>// Mark the first 3 elements of the vector as being initialized. </span>unsafe </span>{ </span> v.</span>set_len</span>(</span>3</span>); </span>} </span>assert_eq!(&v, &[</span>0</span>, </span>1</span>, </span>2</span>]); </span></code></pre> This API fills much the same role as the ReadBuf</code> type did in the first solution, but without requiring a new type to be introduced. Usage of uninitialized vectors becomes a lot closer to the current Read::read</code> behavior:</p> // Read into uninit memory. </span>let mut</span> buf = Vec::with_capacity(</span>1024</span>); </span>let</span> read = file.</span>read_buf</span>(&</span>mut</span> buf)?; </span>let</span> data = buf[..read]; </span> </span>// Read into init memory </span>let mut</span> buf = [</span>0</span>; </span>1024</span>]; </span>let</span> read = file.</span>read</span>(&</span>mut</span> buf)?; </span>let</span> data = buf[..read]; </span></code></pre> Implementers of "leaf" IO types (e.g. File</code>, etc.) could use this method to implement a reader which reads directly into uninitialized memory. The default implementation of read_buf</code> could become something like this:</p> impl </span>Read </span>for </span>MyReader { </span> </span>fn </span>read_buf</span>(&</span>mut </span>self</span>, </span>buf</span>: &</span>mut </span>Vec<</span>u8</span>>) -> io::Result<</span>usize</span>> { </span> </span>// zero-fill the unintialized memory </span> </span>let</span> uninit = buf.</span>spare_capacity_mut</span>(); </span> mem::swap(uninit, &</span>mut </span>MaybeUninit::zeroed()); </span> </span>unsafe </span>{ buf.</span>set_len</span>(buf.</span>capacity</span>()) }; </span> </span> </span>// read the data into the buffer </span> </span>self</span>.</span>read</span>(buf) </span> } </span>} </span></code></pre> It's worth pointing out that the name read_buf</code> at this point doesn't convey the intent particularly well anymore. A name such as read_to_capacity</code> might more closely match our intent. To my knowledge: "Read into the uninitialized bytes of a Vec</code>, but don't grow beyond it"</em>, doesn't have any precedent in the stdlib. The closest I can think of is Vec::fill</code></a>, but it's not quite it. If we choose to use this API, method naming and documentation is something we should take a more closer look at.</p> Even reading into the stack becomes possible if we use Vec::with_capacity_in</code></a> backed by a stack allocator. The ergonomics of that aren't ideal yet, but that's expected to significantly improve over time 3</a></sup>.</p> ^{3</sup> I know folks are actively looking at the ergonomics of this, and am excited for some of the designs I've seen.</p> </div> As we mentioned, the main benefit of this approach is that it doesn't introduce a new type and as such maps nicely to existing patterns. Compared to the first solution we have better ergonomics when reading into the heap, and slightly worse ergonomics when reading into the stack. Overall I think this approach is promising, and I prefer it over the first solution.</p>}Solution 3: specialization</h2> Disclaimer: I'm by no stretch an authority on dynamic dispatch or specialization. It might be that I got details wrong, or misunderstood limitations. This section is "best effort", and am happy to update it if turns out I got something wrong.</em></p> Going back to our earlier example, an ideal solution would be if instead of defining a new read_buf</code> methods we could keep using the read</code> method instead:</p> // Read into uninit memory. </span>let mut</span> buf = Vec::with_capacity(</span>1024</span>); </span>let</span> read = file.</span>read</span>(&</span>mut</span> buf)?; </span>// note: `read`, not `read_buf` </span>let</span> data = buf[..read]; </span></code></pre> The way to do this in Rust would be using specialization</em>. Specialization is still considered "unstable", and has not yet been fully formed. So don't expect the code below to work anytime soon, if ever. But if we squint a little, we could imagine a trait could be defined as follows:</p> pub trait </span>Read { </span> </span>// The "default" implementation. </span> default </span>fn </span>read</span>(&</span>mut </span>self</span>, </span>buf</span>: &</span>mut</span> [</span>u8</span>]) -> Result<</span>usize</span>>; </span> </span> </span>// Implementation specialized for `&mut Vec<u8>` </span> </span>fn </span>read</span>(&</span>mut </span>self</span>, </span>buf</span>: &</span>mut </span>Vec<</span>u8</span>>) -> Result<</span>usize</span>> { </span> </span>// zero-fill the unintialized memory </span> </span>let</span> uninit = buf.</span>spare_capacity_mut</span>(); </span> mem::swap(uninit, &</span>mut </span>MaybeUninit::zeroed()); </span> </span>unsafe </span>{ buf.</span>set_len</span>(buf.</span>capacity</span>()) }; </span> </span> </span>// read the data into the buffer </span> </span>self</span>.</span>read</span>(buf.</span>as_slice</span>()) </span> } </span>} </span></code></pre> In this example we define the trait Read</code> with a "default" implementation for &mut [u8]</code>, and a built-in specialization for &mut Vec<u8></code>. The specialization needs to be built directly into the trait definition, to ensure that &mut Vec<u8></code> will never default to being interpreted as an zero-length slice. Trait implementers would be expected to provide their own specialization for &mut Vec<u8></code>, to make use of the ability to write directly into uninitialized memory.</p> Another thing that's unclear about specializations is the interactions with semver. In our example we're now using Vec::capacity</code> rather than Vec::len</code> to determine how many bytes to write into the vector. All observable changes in behavior have the potential to be breaking</a>, so any modification to this behavior should always use crater to determine impact. But it doesn't seem like we would break any intended use of the APIs.</p> At first glance this interface might seem ideal. It's the same interface that just works</em> depending on what's passed. But as remarked in RFC 2930: read_buf</a>, this approach has issues:</p> It must be compatible with dyn Read. Trait objects are used pervasively in IO code, so a solution can't depend on monomorphization or specialization.</li> </ol> </blockquote> Further elaboration on this point exists in this document</a> and this interview</a>. My interpretation of these materials is: "Specialization is not dyn safe, so if we add a specialization dyn Read</code> would no longer be possible". That would mean the if Read</code> specialized as we showed, the following code would fail to compile:</p> // This alias would cause an error because `Read` is not dyn safe. </span>type </span>MyType = Box<dyn Read>; </span></code></pre> Intuitively I would've expected this to be possible if we wanted it to, even if it's currently not implemented in the compiler. But my understanding of dyn</code> is limited compared to the folks who've written the docs, so I trust there are good reasons why they marked it as a non-option.</p> That said though; the constraints around dyn</code> in Rust are currently being reworked in order to enable things like an AsyncIterator</code> with async fn next</code></a> to work. Which makes me wonder: is the interaction between dyn</code> and specialization something which can be reworked as well?</p> I asked my colleague Sy</a> whether combining dynamic dispatch with specialization is possible in C++, and it appears it is! They walked me through this C++ example</a> which combines a dynamic dispatch based on the class it's called on and whether a vec</code> or span</code> (slice) is passed. Even though the code is very different from Rust, the functionality matches how we would expect it to work.</p> If we think about the wider implications of this, it seems not great</em> if specializations and dynamic dispatch would inherently remain mutually exclusive. Both are incredibly useful, and if they can't be used together that seems incredibly limiting.</p> Summary</h2> In this post we've shown three examples for reading into uninitialized memory in Rust. As I mentioned throughout the post: it'd be great if we could make reading into unitialized memory as close as possible to reading into initialized memory. And I think in particular the second, and third examples have a lot of potential.</p> I was motivated to write this post after conversations with nrc</a> last week. I remembered reading Amanieu's</a> comment about spare_capacity_mut</code> on the tracking issue for RFC 2930</a>, and started thinking of what it could look like if we used that instead of ReadBuf</code>.</p> In terms of timelines, I don't believe we're under great pressure to land the features described in this post into the stdlib. The performance benefits can be significant under certain workloads; but as we're collectively moving towards completion-based IO, traits such as AsyncBufRead</code></a> will become more relevant</a>. Because these traits manage buffers internally, uninitialized memory never needs to cross trait boundaries and this post doesn't apply.</p> I hope I was able to provide some insight on how we can enable reading into uninitialized memory for (async) IO traits. If you liked this post and would like to see my cats, you can follow me on Twitter</a>.</p> Thanks to Sy Brand</a> for showing me how dynamic dispatch and specialization interact in C++, and nrc</a> for helping review this post.</em></p> Async Cancellation I Wed, 10 Nov 2021 00:00:00 +0000 Sometimes we start things but decide midway through that we would prefer to rather not be doing them. That process is sometimes referred to as cancellation. Say we accidentally clicked "download" on a large file in the browser. We should have a way to tell the computer to stop downloading it.</p> When the async foundations WG was researching user stories</a> earlier this year, async cancellation came up repeatedly. It's one of those things that's important to have, but can be tricky to reason about. This is not helped by the fact that little has been written about it, so I figured I might try to make a dent in that by writing a deep-dive on the topic.</p> In this post we'll look at async Rust's async primitives, and cover how cancellation works for those primitives today. We'll then proceed to look at ways in which we can ensure we do not end up with dangling resources. And finally we'll take a look at what the current direction of async Rust means for async cancellation. Sounds like a plan? Good, let's dive in!</p> Tasks and Futures</h2> For the purpose of this post we need to distinguish between two types of async primitives in Rust: futures and tasks 1</a></sup>.</p> ^{1</sup> Arguably "Stream" / "AsyncIterator" is a third async primitive, but everything we say about the Future</code> type applies to Stream</code> as well, so we're considering those the same in this post.</p> </div>} Futures</strong> represent the core building block of async Rust, and exist as part of the Rust language and stdlib. A future is something that when .await</code>ed becomes a value later</em>. By design it does nothing unless .await</code>ed, and must be driven to completion by some other, external loop (example of such a loop</a>).</p> </li> Tasks</strong> are not yet part of the Rust language or stdlib, and represent a managed 2</a></sup> piece of async execution backed by an asynchronous runtime 3</a></sup>. Tasks often allow for async code to become multi-threaded: it's common for executors to schedule tasks across multiple threads, and even move them between threads after they've started executing. A task does not need to be .await</code>ed before it starts executing, and the future it returns is just a way to get the return value of the task once it finishes.</p> </li> </ul> ^{2</sup> "managed" is used throughout this post as: "is scheduled on a runtime". This typically comes with the requirement futures are 'static</code> (the future does not hold any borrows), and when scheduled on a multi-threaded runtime often, but not always, requires futures be Send</code> as well (the future is thread-safe).</p> </div>}^{3</sup> You can think of Tasks as being something akin to lightweight threads, managed by your program rather than you operating system.</p> </div>}Many languages, including JavaScript, use equivalents to Rust tasks rather than Rust futures as their core building blocks 4</a></sup>. This is convenient because only a single type of async building block is exposed, and the language runtime optimizers can figure out how to speed things up if needed. But in Rust we unfortunately can't rely on that, so we manually distinguish between unmanaged (futures) and managed (tasks) primitives.</p> ^4</sup>JavaScript's counterpart to Rust's tasks is Promise</code></a>. It starts being executed the moment it's instantiated, rather than the moment it's await</code>ed. My understanding is that C#'s Task<T></code></a> works much the same way.</p> </div> Cancelling Futures</h2> Cancellation allows for a future to stop doing work early, when we know we're no longer interested in its result. Futures in Rust can be cancelled</em> at one of two points. For simplicity most our examples in this post are going to use sleeping and printing operations; where in the real world we'd probably be talking about file/network operations and data processing instead.</p> 1. Cancel a future before it starts executing</strong></p> Here we create a drop guard, pass it to an async function which returns a future, and then drop the future before we .await</code> it:</p> use </span>std::time::Duration; </span> </span>struct </span>Guard; </span>impl </span>Drop </span>for </span>Guard { </span> </span>fn </span>drop</span>(&</span>mut </span>self</span>) { </span> println!("</span>2</span>"); </span> } </span>} </span> </span>async </span>fn </span>foo</span>(</span>guard</span>: Guard) { </span> println!("</span>3</span>"); </span> task::sleep(Duration::from_secs(</span>1</span>)).await; </span> println!("</span>4</span>"); </span>} </span> </span>fn </span>main</span>() { </span> println!("</span>1</span>"); </span> </span>let</span> guard = Guard {}; </span> </span>let</span> fut = </span>foo</span>(guard); </span> </span>drop</span>(fut); </span> println!("</span>done</span>"); </span>} </span></code></pre> This prints:</p> > 1 </span>> 2 </span>> done </span></code></pre> Our Guard</code> type here will print when its destructor (Drop</code>)</a> is run. We never actually execute the future, but the destructor is still run because we passed the value to the async function. This means the first cancellation point of any future is immediately after instantiation before the async functions's body has run. Meaning not all cancellation points are necessarily demarked by .await</code>.</p> 2. Cancel a future at an await point</strong></p> Here we create a future, poll it exactly once, and then drop it:</p> use </span>std::{ptr, task}; </span>use </span>async_std::task; </span>use </span>std::time::Duration; </span> </span>async </span>fn </span>foo</span>() { </span> println!("</span>2</span>"); </span> task::sleep(Duration::from_secs(</span>1</span>)).await; </span> println!("</span>3</span>"); </span>} </span> </span>let mut</span> fut = Box::pin(</span>foo</span>()); </span>let mut</span> cx = </span>empty_cx</span>(); </span> </span>println!("</span>1</span>"); </span>assert!(fut.</span>as_mut</span>().</span>poll</span>(&</span>mut</span> cx).</span>is_pending</span>()); </span>drop</span>(fut); </span>println!("</span>done</span>"); </span> </span>/// Create an empty "Waker" callback, wrapped in a "Context" structure. </span>/// How this works is not particularly important for the rest of this post. </span>fn </span>empty_cx</span>() -> task::Context { ... } </span></code></pre> > 1 </span>> 2 </span>> done </span></code></pre> Fundamentally, you can think of .await</code> as marking a point where cancellation may occur.</strong> Where keywords like return</code> and ?</code> mark points where the function may return a value, .await</code> marks a location where the function's caller may decide the function should not run any further. But importantly: in all cases destructors will be run, allowing resources to be cleaned up.</p> Futures cannot be cancelled in between .await</code> calls, or after the last .await</code> call. We also don't yet have a design for "async Drop</code>" so we also can't say anything meaningful yet about how how that will interact with cancellation.</p> Cancelling Tasks</h2> Because tasks aren't standardized in Rust yet, cancellation of tasks isn't either. And unsurprisingly different runtimes have different ideas on how to cancel a task. Both async-std</code> and tokio</code> share a similar task model. I'm most comfortable with async-std</code> 5</a></sup>, so let's use that as an example:</p> ^{5</sup> Probably because I co-authored the project.</p> </div>}use </span>async_std::task; </span> </span>let</span> handle = task::spawn(async { </span> task::sleep(Duration::from_secs(</span>1</span>)).await; </span> println!("</span>2</span>"); </span>}); </span> </span>println!("</span>1</span>"); </span>drop</span>(handle); </span>// Drop the task handle. </span>task::sleep(Duration::from_secs(</span>2</span>)).await; </span>println!("</span>done</span>"); </span></code></pre> > 1 </span>> 2 </span>> done </span></code></pre> Here we dropped the handle, but the task continued to run in the background. This is because the async-std</code> runtime uses "detach on drop" semantics for tasks. This is the same in the tokio</code> runtime. In order to cancel a task, we need to manually call a method on the handle. For async-std</code> this is JoinHandle::cancel</code></a>, and for tokio</code> this is JoinHandle::abort</code></a>:</p> use </span>async_std::task; </span> </span>let</span> handle = task::spawn(async { </span> task::sleep(Duration::from_secs(</span>1</span>)).await; </span> println!("</span>2</span>"); </span>}); </span> </span>println!("</span>1</span>"); </span>handle.</span>cancel</span>().await; </span>// Cancel the task handle </span>task::sleep(Duration::from_secs(</span>2</span>)).await; </span>println!("</span>done</span>"); </span></code></pre> > 1 </span>> done </span></code></pre> We can see that if we call JoinHandle::cancel</code>, the task is cancelled at that point, and the number 2</code> is no longer printed.</p> Propagating cancellation for futures</h2> In Async Rust cancellation automatically propagates for futures. When we stop polling a future, all futures contained within will in turn also stop making forward progress. And all destructors</a> within them will be run. In this example we'll be using the FutureExt::timeout</code></a> function from async-std</code>, which returns a Result<T, TimeoutError></code>.</p> use </span>async_std::prelude::; </span>use </span>async_std::task; </span>use </span>std::time::Duration; </span> </span>async </span>fn </span>foo</span>() { </span> println!("</span>2</span>"); </span> </span>bar</span>().</span>timeout</span>(Duration::from_secs(</span>3</span>)).await; </span> println!("</span>5</span>"); </span>} </span> </span>async </span>fn </span>bar</span>() { </span> println!("</span>3</span>"); </span> task::sleep(Duration::from_secs(</span>2</span>)).await; </span> println!("</span>4</span>"); </span>} </span> </span>println!("</span>1</span>"); </span>foo</span>().</span>timeout</span>(Duration::from_secs(</span>1</span>)).await; </span>println!("</span>done</span>"); </span></code></pre> > 1 </span>> 2 </span>> 3 </span>> done # `4` and `5` are never printed </span></code></pre> The tree of futures above can be expressed as the following graph:</p> main </span> -> foo (times out after 1 sec) </span> -> bar (times out after 3 secs) </span> -> task::sleep (wait for 2 secs) </span></code></pre> Because foo</code> is timed out and dropped before bar</code> has the chance to complete, not all numbers get to be printed. bar</code> is dropped before it has the opportunity to complete, and all of its resources are cleaned up when its destructors run.</p> This means that in order to cancel a chain of future, all we need to do is to drop it</em>, and all resources will in turn be cleaned up. Whether we're dropping a future by hand, calling a timeout method</a> on a future, or racing multiple futures</a> — cancellation will propagate, and resources will be cleaned up.</p> Propagating cancellation for tasks</h2> As we showed earlier, simply dropping a task handle is not enough to cancel a task in most runtimes. We need to explicitly invoke a cancellation method to cancel the task. This means tasks aren't automatically propagated.</p> In hindsight this was probably a mistake. More specifically, this was likely my</em> mistake. async-std</code>'s JoinHandle</code> was the first in the ecosystem, and I pushed that we should model it using the "detach-on-drop" behavior directly after std::thread::JoinHandle</code>. What I didn't account for though, is that threads cannot be cancelled externally: std::thread::JoinHandle</code> doesn't, and may likely never have a cancel</code> method 6</a></sup>.</p> ^{6</sup> The way a thread can be cancelled from the outside is by passing a channel to the thread, and cancelling when a message is received. Unlike async Rust, threads don't have .await</code> points which act as natural stopping points. So instead threads have to opt-in to cancellation by manually listening for signals.</p> </div> Failing to propagate cancellation means that if we cancel a tree of work, we may end up with dangling tasks</strong> that continue when we really don't want to. Rather than having a cancel</code> method which allows us to manually opt-in to cancellation propatation (more on that later), cancellation propagation should be opt-out by default.</p>}Luckily we don't need to guess what a runtime with "cancellation propagation is opt-out" behavior would look like. The async-task</code></a> executor, and in turn smol</code></a> runtime do exactly that.</p> smol::block_on(async { </span> println!("</span>1</span>"); </span> </span>let</span> task = smol::spawn(async { </span> println!("</span>2</span>"); </span> }); </span> </span>drop</span>(task); </span> println!("</span>done</span>") </span>}); </span></code></pre> > 1 </span>> done </span></code></pre> Patching cancellation propagation</h2> Unfortunately only smol</code> propagates cancellation across tasks, and it would be a breaking change to modify the cancellation propagation behavior of JoinHandle</code> in other runtimes.</p> Users of runtimes can still ensure that cancellation will always correctly be propagated by creating a custom spawn</code> function that contains a drop guard like so:</p> use </span>tokio::task; </span>use </span>core::future::Future; </span>use </span>core::pin::Pin; </span>use </span>core::task::{Context, Poll}; </span> </span>/// Spawn a new tokio Task and cancel it on drop. </span>pub fn </span>spawn</span><T>(</span>future</span>: T) -> Wrapper<</span>T::</span>Output> </span>where </span> T: Future + Send + </span>'static</span>, </span> </span>T::</span>Output: Send + </span>'static</span>, </span>{ </span> Wrapper(task::spawn(future)) </span>} </span> </span>/// Cancels the wrapped tokio Task on Drop. </span>pub struct </span>Wrapper<T>(task::JoinHandle<T>); </span> </span>impl</span><T> Future </span>for </span>Wrapper<T>{ </span> </span>type </span>Output = Result<T, task::JoinError>; </span> </span>fn </span>poll</span>(</span>mut </span>self</span>: Pin<&</span>mut Self</span>>, </span>cx</span>: &</span>mut </span>Context<'_>) -> Poll<</span>Self::</span>Output> { </span> </span>unsafe </span>{ Pin::new_unchecked(&</span>mut </span>self</span>.</span>0</span>) }.</span>poll</span>(cx) </span> } </span>} </span> </span>impl</span><T> Drop </span>for </span>Wrapper<T> { </span> </span>fn </span>drop</span>(&</span>mut </span>self</span>) { </span> </span>// do `let _ = self.0.cancel()` for `async_std::task::Task` </span> </span>self</span>.</span>0.</span>abort</span>(); </span> } </span>} </span></code></pre> This wrapper can be adapted to work for async-std</code> as well, and ensures that cancellation propagates across task bounds.</p> Structured concurrency</h2> Given we're talking a lot about propagating cancellation and trees in this post, we probably should mention the concept of "structured concurrency"</a>.</p> In order for async Rust to be structurally concurrent I think of tasks as having the following requirements:</p> Ensure 7</a></sup> sure child tasks don't outlive their parents.</li> If we cancel a parent task the child task should be cancelled too.</li> If a child task raises an error, the parent task should be able to act on it.</li> </ol> ^{7</sup> More on whether we can actually ensure this later on in this post.</p> </div> This structure makes it so errors don't accidentally go ignored, or we fail to cancel tasks further down in the tree. It's the basis for effectively implementing other mechanisms on top, such as retries, limits, supervisors, and transactions.</p>}Cancelling a group of tasks</h2> Cancellation becomes more difficult to implement when we're dealing with a stream of tasks, each of which needs to be spawned on the runtime. Currently a lot of async code just spawns tasks, detaches them, and logs if case of an error:</p> // The async-std "echo tcp server" example. </span>use </span>async_std::io; </span>use </span>async_std::net::{TcpListener, TcpStream}; </span>use </span>async_std::prelude::; </span>use </span>async_std::task; </span> </span>// Listen for new TCP connections on port 8080. </span>let</span> listener = TcpListener::bind("</span>127.0.0.1:8080</span>").await?; </span> </span>// Iterate over the incoming connections, and move each task to a multi-threaded </span>// runtime. </span>let mut</span> incoming = listener.</span>incoming</span>(); </span>while let </span>Some(stream) = incoming.</span>next</span>().await { </span> task::spawn(async { </span> </span>// If an error occurs, log it to stderr. </span> </span>if let </span>Err(err) = </span>run</span>(stream).await { </span> eprintln!("</span>error: </span>{}</span>", err); </span> } </span> }); </span>} </span> </span>// The main logic of our listen loop. This is a simple echo server. </span>async </span>fn </span>run</span>(</span>stream</span>: io::Result<TcpStream>) -> io::Result<()> { </span> </span>let</span> stream = stream?; </span> </span>let </span>(reader, writer) = &</span>mut </span>(&stream, &stream); </span> io::copy(reader, writer).await?; </span> Ok(()) </span>} </span></code></pre> But if we're trying to do Structured Concurrency</a> correctly, we need to ensure cancellation propagates to those spawned tasks too. In order to do so we need to introduce a new async primitive to Rust: the TaskGroup</code>8</a></sup>.</p> ^8</sup>This terminology is borrowed from Swift, but is similar-ish to how crossbeam-scope</code></a> works (n threads managed by a central point). However unlike crossbeam-scope</code>, the name concerns itself less with how the lifetimes flow, and more how you can reason about how it should be used.</p> </div> To my knowledge no runtimes currently support this out of the box, but crates for this exist on crates.io. One example of such a crate is task-group</code></a>, authored by the Fastly WASM group. Functionally it allows creating a group of tasks which act as a single unit. If a task errors or panics, all other tasks are cancelled. And when the TaskManager</code> is dropped, all currently running tasks are cancelled.</p> Task grouping (including task scopes) is a topic that needs its own blog post. But if you're looking to apply cancellation propagation to a stream of items, at least now you know this is the primitive that enables that to work.</p> halt-safety</h2> So far we've talked a fair bit about cancellation in this post. Now that we know that the existence of .await</code> means that our function might exit, it's natural to ask: "If our Future's state machines can be cancelled at any state, how do we ensure they function correctly?"</em></p> When we're using futures created through async/.await</code>, cancellation at .await</code> points will functionally act no different than early returns through the try (?</code>) operator.</strong> In both cases function execution is halted, destructors are run, and resources are cleaned up. Take the following example;</p> // Regardless of where in the function we stop execution, destructors will be </span>// run and resources will be cleaned up. </span>async </span>fn </span>do_something</span>(</span>path</span>: PathBuf) -> io::Result<Output> { </span> </span>// 1. the future is not guaranteed to progress after instantiation </span> </span>let</span> file = fs::open(&path).await?; </span>// 2. `.await` and 3. `?` can cause the function to halt </span> </span>let</span> res = </span>parse</span>(file).await; </span>// 4. `.await` can the function to halt </span> res </span>// 5. execution has finished, return a value </span>} </span></code></pre> do_something</code> has 5 distinct points where it can halt execution and destructors are run. It doesn't matter whether .await</code> triggers a cancellation, ?</code> returns an error, or our function exits as expected. Our destructors don't need to care, they only need to concern themselves with is ensuring they release whatever resources they're holding onto 9</a></sup>.</p> ^{9</sup> "Clean up resources regardless of what triggered the function to halt" is what I'm referring to as "halt-safety". I'm not in love with the term, but I needed to find a way to give a name to this group of mechanisms. I hope the term is clear enough.</p> </div>}Things change a bit when we talk about Futures created using manual Poll</code></a> state machines. Within a manual state machine we don't call .await</code>; instead we make forward progress by manually calling Future::poll</code> or Stream::poll_next</code>. Similar when dealing with async/.await</code> we want to ensure that destructors are run should any of poll_</code>, ?</code>, or return</code> run. Unlike async/.await</code> futures, resiliency against our function halting isn't automatically provided for us. Instead manually authored futures need to implement the resiliency that async/.await</code> futures rely on.</strong></p> I like to think about a futures call graph like a tree. There are the leaf nodes in the tree, which need to guarantee cancellation is handled correctly. And there are branch nodes in the tree, which are all created through async/.await</code> and rely on the leaf nodes correctly implementing resource cleanups.</p> Kind of like how in non-async Rust we don't really need to think about releasing file handles or memory since we rely on that just working out of the box. The only time we really need to think about how to release resources, is when we're implementing primitives like TcpStream</code> or Vec</code> ourselves.</p> Should tasks be detachable?</h2> We've talked quite a bit about the importance of cancellation propagation for tasks, perhaps to the point that you might now be wondering whether tasks should be detachable at all. Unfortunately destructors in Rust are not guaranteed to run</a>, so we can't can't actually stop people from passing JoinHandle</code>s to mem::forget</code> in order to detach their tasks.</p> That does not mean that we should make detaching tasks easy though. For example: MutexGuard</code> can be passed to mem::forget</code> too, but we don't expose a method for that directly on MutexGuard</code> because doing so causes the lock to be held forever. Depending on how we feel about dangling tasks, we may want to use API design to disincentivize people from going into unwanted states.</p> An async trait which can't be cancelled?</h2> There has been talk in the Async Foundations WG about potentially creating a new async trait that would create a future which guarantees it must be run to completion (e.g. "can't be cancelled"). Our existing core::future::Future</code> trait would inherit from that trait, and add the additional guarantee that it can in fact be cancelled.</p> I've seen two main motivations for going in this direction:</p> FFI compatibility with C++'s async system, where failing to run a "C++ Future" to completion is undefined behavior.</li> It would make it easier for non-experts to author async Rust by making "cancellation" something you don't need to learn about up front.</li> </ol> This is not the right post to dig into the first point, but the second point certainly is interesting for us here. While I agree this may indeed pose a hurdle for people writing async today, I don't believe it will going forward because of the work we're doing to eliminate the need to manually author Poll</code> state machines already. Work is currently ongoing to for example change the signature of Stream</code> from fn poll_next</code> to async fn next</code>. Similarly the working group is working on adding async traits and async closures, all with the goal to reduce the need to author futures by hand.</p> As we've observed, handling cancellation is most difficult when authoring futures by hand. As that is where we need to build the resiliency to halts (through cancellation and errors alike) that the rest of the futures call graph relies upon. If we make it so that manually authored futures are only required to implement the primitives that the rest of the ecosystem relies upon, then the problem has already been solved.</strong></p> To close this out: I think Rust's general approach to ensuring {IO safety</a>, memory safety, halt safety} should be multi-pronged. We should make it so that for the vast majority of operations, operators shouldn't need to reach for Rust's powertools. But for when powertools are definitely the way to go, we should add all the lints and hints we can to ensure they're operated safely 10</a></sup>.</p> ^10</sup>For example, if I ever need to author something using [MaybeUninit</code>], I want the compiler to remind me to author a drop guard</a> when keeping it live past something which may panic. Perhaps someday (:</p> </div> intermediate matching on cancellation?</h2> While editing this post, I heard of a possible third motivation to potentially have an alternate future trait: wanting the ability to match on cancellation. The idea is that we can replace ?</code> with match</code> to handle errors using alternate logic, but we cannot do at a cancellation point for .await</code>.</p> // Using the Try operator to re-throw the error. </span>let</span> string = fs::read_to_string("</span>my-file.txt</span>").await?; </span> </span>/// Manually re-throwing the error. </span>let</span> string = </span>match </span>fs::read_to_string("</span>my-file.txt</span>").await { </span> Err(err) => </span>return </span>Err(err), </span> Ok(s) => s, </span>}; </span> </span>// We can replace the `?` with `match`, but we can't replace `.await` </span>// with anything else. </span></code></pre> The idea is that in conjunction with an un-cancellable future trait, cancellation would instead be performed by sending a signal to the underlying IO resource we're waiting on, which would in turn stop what it's doing and return an io::ErrorKind::Interrupted</code> error that should manually be bubbled up by intermediate futures to where it can be matched on by the caller.</p> This argument may sound appealing: right now we indeed can't destructure .await</code> into a match</code> statement the way we can with ?</code>. So maybe this mechanism would be useful?</p> To dive in a little; let's assume we have the following tree of futures:</p> main </span> -> foo (times out after 1 sec) </span> -> bar (times out after 3 secs) </span> -> task::sleep (wait for 2 secs) </span></code></pre> If we want to handle the timeout of foo (times out after 1 sec)</code> in our main function, we could just match on it in either case:</p> // Current method of handling async cancellation. </span>let</span> dur = Duration::from_secs(</span>1</span>); </span>let</span> string = </span>match </span>fs::read_to_string("</span>my-file.txt</span>").</span>timeout</span>(dur).await { </span> Err(err) => panic!("</span>timed out!</span>"), </span> Ok(res) => res?, </span>}; </span> </span>// Runtime-signal cancellation. </span>let</span> dur = Duration::from_secs(</span>1</span>); </span>let</span> string = </span>match </span>fs::read_to_string("</span>my-file.txt</span>").</span>timeout</span>(dur).await { </span> Err(ErrorKind::Interrupted) => panic!("</span>timed out!</span>"), </span> Err(err) = </span>return</span> err, </span> Ok(s) => s, </span>}; </span></code></pre> In practice both approaches have incredibly similar semantics at the calling side. The main difference is that in the runtime-signal approach we may never hit the error path. In fact, there is no longer a guarantee that the inner futures propagate the cancellation. They can instead choose to ignore the cancellation and (erroneously) attempt to retry. Retries should always be scheduled alongside timeouts; that way if we timeout once we can retry again. If the two are de-coupled, retries beyond the first will not have an associated timeout, and risk hanging forever. This is undesireable, and something we should steer people away from. And our existing semantics already achieve that.</p> There may be reasons why we might want to detect cancellation on intermediate futures, for example for the purpose of providing internal logging</a>. But this is already possible by combining completion status tracking</a> with Drop</code> guards.</p> This is further complicated by the fact that we're now overloading io::ErrorKind::Interrupted</code> to carry cancellations from both the operating system, and trigger in user space. Errors cause by the operating system should be retried in-place. But errors cause by users should always bubble up. We can no longer tell them apart.</p> Another issue is that in order to uniformly support cancellation of futures futures must now need to carry an io::Result</code> in their return type. We'd need to rewrite I/O primitives such as task::sleep</code></a> to be fallible just for this purpose. It's not the worst; but it is reminescent of the Futures 0.1 days, where futures had always had to be fallible</a>.</p> Overall I do think both approaches are roughly comparable. But because the cancellation-through-signals variant enables cancellations to be ignored (accidental, or otherwise), it exposes a set of footguns to users of async that our current system does not have 11</a></sup>.</p> ^{11</sup> This is probably worth an entire blog post on its own. But I have so many posts on in-progress already that I figured I'd add it in here.</p> </div>}Defer blocks?</h2> The mechanisms to guard against halt-safety are similar to those for unwind safety</a>: create a drop guard to ensure we run our destructors. Manually writing these can get pretty verbose</a>.</p> Languages such as Swift and Go provide a language-level solution to this, in the form of the defer</code> keyword. This effectively allows users of the language to write an in-line drop guard to create an anonymous destructor. "The defer keyword in Swift: try/finally done right"</a> is a thorough overview of how this works in Swift. But for illustrative purposes, here's an example:</p> func </span>writeLog() { </span> </span>let</span> file = openFile() </span> </span>defer</span> { closeFile(file) } </span> </span> </span>let</span> hardwareStatus = fetchHardwareStatus() </span> </span>guard</span> hardwareStatus != </span>"disaster" </span>else</span> { </span>return </span>/ defer block runs here /</span> } </span> file.write(hardwareStatus) </span> </span> </span>// defer block runs here </span>} </span></code></pre> Rust's scopeguard crate</a> provides a collection of defer</code> macros. But these wouldn't suffice for the example we shared earlier</a>, since we want to maintain access to the data while continuing to access it, and scopeguard::defer</code> doesn't let us</a>. This is because what we want is drop guards to not take ownership of the value until the destructor is run. I believe this is also referred to as late binding</em>. And the best way we could achieve that would be by introducing a language feature.</p> To be clear: I'm not necessarily advocating we introduce defer</code> into Rust. It would add a form of non-linear control flow to the language which could confuse a lot of folks 12</a></sup>. But if we consider {halt, unwind}</code>-safety to be important enough that we want to provide users with better tools; then defer</code> seems like a candidate we may want to explore more closely.</p> ^{12</sup> Folks have told me this from using other languages. I've never actually used defer</code>, so I can't comment on what it's like. But I have done a lot of non-linear control flow by writing callback-heavy JavaScript, and that indeed takes some getting used to.</p> </div>}update (2021-11-14):</em> as folks correctly pointed out, while scopeguard::defer!</code> does not provide provide access to the captured values before dropping, scopeguard::ScopeGuard</code></a> does via the Deref</code> / DerefMut</code> traits. The flexibility of being able to remove the Drop</code> impl of the guard by converting the guard into its inner value makes for a compelling argument that solving in-line drop impls would be better solved through a library addition than through a language item.</p> Cancellation and io_uring</h2> This section was added on 2021-11-17, after the post was initially published.</em></p> Patrick Walton</a> asked the following question /r/rust</a>:</p> One motivation for completion futures that either wasn't mentioned or is an unexpected side effect of point #1 (compatibility with C++) is that async code in C/C++ can take non-owning references into buffers. For example, on Linux if you issue an async read() call using io_uring and you later get cancelled, you have to tell the kernel somehow that it needs to not touch the buffer after Rust frees it. There are ways to do this, such as giving the kernel ownership of the buffers using IORING_REGISTER_BUFFERS, but having the kernel own the I/O buffers can make things awkward. (Async C++ folks have shown me patterns that would require copies in this case.) Have you all given any thought as to the best practices here? It's a tough decision, as the only real solution I can think of involves making poll() unsafe, which is unsavory (NB: not necessarily wrong).</p> </blockquote> Saoirse</a> wrote an excellent overview</a> of how the completion-based Linux io_uring</code> family of kernel APIs</a> interacts with cancellation in Rust. The whole post is worth a read as it covers much of the nuance and safety considerations involved when using io_uring</code> with Rust. But it also directly answers Patrick's question:</p> So I think this is the solution we should all adopt and move forward with: io-uring controls the buffers, the fastest interfaces on io-uring are the buffered interfaces, the unbuffered interfaces make an extra copy. We can stop being mired in trying to force the language to do something impossible.</p> </blockquote> Conclusion</h2> In this post we've looked at how cancellation works for futures and tasks. We've covered how cancellation propagation ought to work, and how we can backport it to existing runtimes. And finally we've covered how to reason about structured concurrency, how cancellation propagation can be applied to groups of tasks, and how async cancellation might interact with async designs currently being drafted.</p> I hope this was informative! You might have noticed the title of this post has a "1" appended to it. I'm planning to post a follow-up to this post covering the design space of triggering cancellation at a distance, and possibly another on task grouping. working on different posts on async concurrency, so expect more on that soon.</p> I've gone and opened a discussion thread on Internals</a>. And if you enjoyed this post, and would like to see whatever I'm dreaming up in a real-time fashion, you can follow me @yoshuawuyts</a>.</p> Update (2021-11-14):</em> Firstyear wrote a follow-up to this post</a> on transactional operations in Rust, and how they interact with (async) cancellation and halt-safety. If you're interested in transactions and rollbacks, I recommend giving it a read!</p> Thanks to: Eric Holk, Ryan Levick, Irina Shestak, and Francesco Cogno for helping review this post prior to publishing. And thanks to Niko Matsakis for walking me through some of the alternate futures (lol) of the future trait.</em></p> Futures Concurrency II: A Trait Approach Thu, 02 Sep 2021 00:00:00 +0000 It's been exactly two years since I wrote about futures concurrency</a>. Some work has happened since, and I figured it would be a good time to recap futures concurrency, and share some of the new developments.</p> In this post we'll establish a base model of async concurrency for Rust. We'll cover ergonomic new APIs to implement said model. And finally we'll describe a plausible path towards inclusion of those APIs in the stdlib in the short-term.</p> Modes of concurrency</h2> Concurrency can be thought of as awaiting multiple things at the same time on the same thread. Generally we can divide the various modes of concurrency up along two axes:</p> Do we want to wait for all outputs, or just the first output?</li> When an error occurs, do we keep going or do we return early ("short circuit")?</li> </ol> As of August 2020, JavaScript has had the following Promise</code> concurrency APIs available on most platforms:</p> </th> Wait for all outputs</strong></th> Wait for first output</strong></th></tr></thead> Continue on error</strong></td> Promise.allSettled</code></a></td> Promise.any</code></a></td></tr> Return early on error</strong></td> Promise.all</code></a></td> Promise.race</code></a></td></tr> </tbody></table> JavaScript covers most concurrency uses really well. Rust has a counterpart to JavaScript's Promise</code> type in Future</code>, but it differs in two key ways:</p> Promise</code>s are always fallible, while Future</code>s have the option not to be.</li> Promise</code>s start executing immediately, while Future</code>s need to be invoked first. 1</a></sup></li> </ol> ^1</sup>In terms of execution, the rough equivalent of a Promise</code> in JavaScript is Task</code></a> in Rust (starts executing immediately on creation). The rough equivalent of a Future</code> in Rust is a "thenable"</a> in JavaScript (starts executing once await</code>ed).</p> </div> For async-std</code></a> we've been experimenting with introducing similar concurrency APIs as those found for JavaScript Promise</code>s as part of our future</code> submodule. For Rust Futures</code> which don't return Result</code> we expose two methods for concurrency:</p> Wait for all outputs</strong></th> Wait for first output</strong></th></tr></thead> Future::join</code></td> Future::race</code></td></tr> </tbody></table> When we add fallability</em> to the mix (e.g. return a Result</code> from the future) async-std</code>'s try_</code> methods for concurrency become available we can compare it 1:1 to JavaScript's APIs:</p> </th> Wait for all outputs</strong></th> Wait for first output</strong></th></tr></thead> Continue on error</strong></td> Future::join</code></td> Future::try_race</code></td></tr> Return early on error</strong></td> Future::try_join</code></td> Future::race</code></td></tr> </tbody></table> The try</code> prefix can be thought of as inverting the semantics of the method. In the case of try_join</code> we no longer wait for all</em> items to complete; on error we'll now exit early and cancel all others. In the case of try_race</code> we no longer wait for the first</em> item to complete; on error we'll keep waiting until we either find a successful item or run out of items.</p> This is the model we wrote about two years ago, and aside from a rename (select</code> -> race</code>) it's remained identical. However despite that, we still haven't attempted to introduce this functionality into the stdlib yet. And that's because of missing ✨ language features ✨.</p> Joining tuples</h2> In my post "future::join</code> and const-eval"</a> I talk about the issue of joining multiple futures:</p> [...] once we start joining more than two futures, the output types become different:</p> </blockquote> let</span> a = future::ready(</span>1</span>u8</span>); </span>let</span> b = future::ready(</span>2</span>u8</span>); </span>let</span> c = future::ready(</span>3</span>u8</span>); </span>assert_eq!(join!(a, b, c).await, (</span>1</span>, </span>2</span>, </span>3</span>)); </span> </span>let</span> a = future::ready(</span>1</span>u8</span>); </span>let</span> b = future::ready(</span>2</span>u8</span>); </span>let</span> c = future::ready(</span>3</span>u8</span>); </span>assert_eq!(a.</span>join</span>(b).</span>join</span>(c).await, (</span>1</span>, (</span>2</span>, </span>3</span>))); </span>// oh no! </span></code></pre> As you can see, each invocation of Future::join returns a tuple. But that means that chaining calls to it starts to nest tuples, which becomes hard to use. And it becomes more nested the more times you chain. Oh no!</p> </blockquote> In the post we explain that until then we'd been solving this 2</a></sup> by using a join!</code> macro, which by nature is variadic 3</a></sup>. However ideally we would be able to write methods which are generic over tuples. Not only would that allow us to write a better join</code> function, it would also save us from having stdlib twelve different PartialEq</code> and PartialOrd</code> impls</a>:</p> ^{2</sup> "this" means having nested return types like Join<Join<Join<A>, B>, C></code> which return tuples in the shape of (a, (b, c))</code>, etc. What we want is to be able to have a single "flat" Join<A, B, C...></code> return type which returns (a, b, c)</code> without any nesting. That's the challenge we're trying to solve.</p> </div>}^{3</sup> "Variadic" means "can support any number of arguments". For example println!</code> can print out any number of arguments. We can't express that using regular Rust functions yet.</p> </div>}Just like const generics</a> solved the a similar issue for arrays, it seems likely like we'll eventually want to solve this for tuples too. But that's unlikely to happen anytime soon (if I'd hazard a guess, I'd say if work starts on this in 2022, then maybe we could have it by 2023?). So for now we need to work around this restriction, and luckily there are a few options.</p> How to expose futures concurrency</h2> In response to "future::join</code> and const-eval"</a>, matthieum pointed out</a> we can invert the API instead. Rather than exposing this functionality as functions scoped under std::future</code>, we could instead extend in-line container types with traits that allow for more fluent composition. I wrote the futures-concurrency</code></a> crate to test out that design, and the design feels incredibly</em> good. For example, this is what it looks like to await multiple similarly-typed futures without any intermediate allocations:</p> use </span>std::future; </span>use </span>futures_concurrency::prelude::; </span> </span>let</span> a = future::ready(</span>1</span>u8</span>); </span>let</span> b = future::ready(</span>2</span>u8</span>); </span>let</span> c = future::ready(</span>3</span>u8</span>); </span>assert_eq!([a, b, c].</span>join</span>().await, [</span>1</span>, </span>2</span>, </span>3</span>]); </span></code></pre> And this is for multiple differently-types futures without any intermediate allocations:</p> use </span>std::future; </span>use </span>futures_concurrency::prelude::; </span> </span>let</span> a = future::ready(</span>1</span>u8</span>); </span>let</span> b = future::ready("</span>hello</span>"); </span>let</span> c = future::ready(</span>3</span>u16</span>); </span>assert_eq!((a, b, c).</span>join</span>().await, (</span>1</span>, "</span>hello</span>", </span>3</span>)); </span></code></pre> But sometimes we want to allocate because we don't know how many futures we're going to want to await in parallel. So it works with Vec<Future></code> as well, similar to futures</code>' join_all</code> function:</p> use </span>std::future; </span>use </span>futures_concurrency::prelude::; </span> </span>let</span> a = future::ready(</span>1</span>u8</span>); </span>let</span> b = future::ready(</span>2</span>u8</span>); </span>let</span> c = future::ready(</span>3</span>u8</span>); </span>assert_eq!(vec![a, b, c].</span>join</span>().await, vec![</span>1</span>, </span>2</span>, </span>3</span>]); </span></code></pre> This feels like the most promising approach so far. Once the right traits are in scope (2024 edition, anyone?), the way to await becomes intuitive. We no longer need to chain calls to .join().join().join()</code> or wrap items in awkward macros. Instead we group futures together, suffix it with our preferred method of concurrency, and then types pop out on the other end. Easy.</p> There are some caveats to this though. The way we've implemented this for tuples is the same way the stdlib implements PartialOrd</code> and PartialEq</code>: by using a macro. However because we're returning custom types from these traits, it means we're currently generating twelve different Join</code> types</a>. Surely there must be a way around this right?</p> Async Traits and TAITs</h2> Currently the futures_concurrency::Join</code> trait is defined like this:</p> trait </span>Join { </span> </span>type </span>Output; </span> </span>type </span>Future: Future<Output = </span>Self::</span>Output>; </span> </span>fn </span>join</span>(</span>self</span>) -> </span>Self::</span>Future; </span>} </span></code></pre> We have two types: the Output</code> type which is what our method will return once awaited. And the Future</code> type which is the future our join</code> method will return that needs to be awaited. However the async foundations WG is working on introducing "async traits" into the language. Once that's in place it's expected we'll be able to rewrite the trait like so:</p> trait </span>Join { </span> </span>type </span>Output; </span> async </span>fn </span>join</span>(</span>self</span>) -> </span>Self::</span>Output; </span>} </span></code></pre> It's expected that all async traits will go this way, since it removes the need to manually implement futures using Pin</code></a> 4</a></sup> (which is a big improvement for ergonomics). However this comes with an issue: How do we manually return futures?</p> ^{4</sup> I consider Pin</code> to be like unsafe {}</code> in that it's essential for Rust to expose this - but under no circumstance should it be required to perform common operations.</p> </div>}The answer for that appears to be TAITS</a> ("Type Alias Impl Trait"). This allows traits to be used in type aliases, and treated as concrete types. Async traits hasn't been RFC'd yet, but I expect we'll want to enable it to work with TAITS once they stabilize. So that would allow us to implement the async trait differently depending on whether we want to name the output type or not:</p> // Implement the trait using `async fn`. </span>impl</span><T> Join </span>for </span>Vec<T> { </span> </span>type </span>Output = Vec<T>; </span> async </span>fn </span>join</span>(</span>self</span>) -> </span>Self::</span>Output; </span>} </span> </span>// Implement the trait using with a named future. </span>type </span>JoinFuture<T> = </span>impl </span>Future<Output = Vec<T>>; </span>impl<T> Join for Vec<T> { </span> </span>type </span>Output = Vec<T>; </span> </span>fn </span>join</span>(</span>self</span>) -> JoinFuture<T>; </span>} </span></code></pre> Perhaps you're seeing where I'm going with this, but for tuples we can then create a shared Join</code> future type for all tuple variants without needing to have variadics in the language:</p> type </span>JoinFuture<T> = </span>impl </span>Future<T>; </span> </span>// Implement `Join` for tuple `(A, B)`. </span>impl<T, </span>const</span> N: </span>usize</span>> Join for (A, B) </span>where </span> A: Future, </span> B: Future, </span>{ </span> </span>type </span>Output = (A::Output, B::Output); </span> </span>fn </span>join</span>(</span>self</span>) -> JoinFuture<</span>Self::</span>Output> { ... } </span>} </span> </span>// ...and repeat for all other tuples (likely using a macro). </span>impl</span><T, </span>const</span> N: </span>usize</span>> JoinTrait </span>for</span> (A, B, C) { ... } </span>impl</span><T, </span>const</span> N: </span>usize</span>> JoinTrait </span>for</span> (A, B, C, D) { ... } </span>impl</span><T, </span>const</span> N: </span>usize</span>> JoinTrait </span>for</span> (A, B, C, D, E) { ... } </span></code></pre> And once we have variadics as part of the language, we could migrate the implementations to use that instead:</p> type </span>JoinFuture<T> = </span>impl </span>Future<T>; </span> </span>// Implement `Join` for all tuples of two or more items. </span>impl<A, B, </span>const</span> N: </span>usize</span>> Join for (A, B) </span>where </span> A: Future, </span> B...: Future, </span>// <- Entirely made up variadic syntax </span>{ </span> </span>type </span>Output = (A::Output, B::Output); </span> </span>fn </span>join</span>(</span>self</span>) -> Join<</span>Self::</span>Output>; </span>} </span></code></pre> It's important to emphasize though that the semantics of TAITS have not yet been stabilized, and are subject to change. But the introduction of TAITS may also have implications for the rest of the language; for example bringing inference to manual trait implementations too. 5</a></sup></p> ^{5</sup> An example of this could be impl Iterator for Foo { fn next(&mut self) -> Option<i32> { ... } } }</code>. The associated type Item</code> can directly be inferred from fn next</code>'s return type. If this was allowed, it would bring the semantics of manual trait implementations closer to those done via TAITs.</p> </div>}Road to std</h2> Despite what you might think after the last section, we don't need to wait until we have async traits, TAITS, or variadics in the stdlib to provide a good experience for async concurrency. Instead we can start small, and as more language features stabilize, incrementally expose more functionality.</p> The most basic form in which we could introduce Join</code> into the stdlib would be by making it available for implementation within the stdlib-only using the C-SEALED future-proofing pattern</a>:</p> trait </span>Join: private::Sealed { </span> </span>type </span>Output; </span> </span>type </span>Future: Future<Output = </span>Self::</span>Output>; </span> </span>fn </span>join</span>(</span>self</span>) -> </span>Self::</span>Future; </span>} </span> </span>mod </span>private { </span> </span>pub</span>(</span>super</span>) </span>trait </span>Sealed {} </span> </span> </span>pub</span>(</span>super</span>) </span>struct </span>Join2 { ... } </span> </span>impl </span>Future </span>for </span>Join2 { ... } </span> </span>impl </span>super::Join </span>for</span> (A, B) { </span>type </span>Future = Join2<A, B>; ... } </span> </span> </span>// ...etc </span>} </span></code></pre> Because in addition to having the trait be sealed, the returned futures are private too. The only way to reference the associated futures is through impl Future</code>. Which lines up perfectly with async traits, which would allow us to drop the Seal</code> and publicly expose the trait as core::future::Join</code> like this:</p> trait </span>Join { </span> </span>type </span>Output; </span> async </span>fn </span>join</span>(</span>self</span>) -> </span>Self::</span>Output; </span>} </span></code></pre> This version still only allows storing the future as impl Future</code>, but once TAITs land can be extended to named futures instead. And then eventually with variadics make the final jump to feature completedness.</p> In summary: if we wanted to, we could start implementing a fully forward compatible version of Join</code> and related APIs in the stdlib today</em>!</p> Conclusion</h2> In this post we've established a basic model for concurrently executing multiple futures. We've described an ergonomic new API, prototyped in the futures_concurrency</code> crate which enables concurrent execution of futures in a more ergonomic way than was previously possible. And finally we've shown a path for gradually introducing these APIs into the stdlib in a forward-compatible manner.</p> For me, the most important point of this post is to show that it's possible to create Futures concurrency APIs which feel natural to use, and can be called with little effort. The exact shape of the traits is less important, and something we can improve on over time. We've shown that we can introduce these methods in the stdlib in a forward-compatible manner, giving us time to polish and iterate on these traits as more language features become available.</p> One mode of execution we didn't cover in this post is a variation on race</code> in which multiple futures are resolved concurrently, and values are yielded as soon as they're resolved. This is what the futures_rs::select!</code> macro is commonly used for. In a future installment of this series we'll take a look at how we could potentially bring that functionality to the stdlib as well.</p> Another thing which we haven't covered yet is how to introduce the try</code> variants of the APIs. The obvious choice would be to introduce a TryJoin</code> trait, but that may not quite be the right fit for what we're trying to do. In a future installment we'll look more closely at introducing the try</code> variants.</p> My plan for the futures-concurrency</code> crate is to keep working on it, and bring it to a state where all the APIs shared in this post have been implemented. Once that's proven to work, and we're sure that this method is forward compatible, we can consider proposing this for addition to the stdlib.</p> But that's something for the future. For now I hope you enjoy the futures-concurrency</code> crate, and I'd be keen to hear what folks think!</p> Special thanks to: Sy Brand</a> for help with understanding variadics in C++, Scott McMurray</a> for elaborating on TAITs, and Ryan Levick</a> and Eric Holk</a> for reviewing this post prior to publishing.</em></p> Async overloading Tue, 24 Aug 2021 00:00:00 +0000 If the stdlib were to expose APIs for async IO, how should it be namespaced?</p> // Like this? </span>use </span>std::async_io::remove_file; </span> </span>// Like this? </span>use </span>std::io::async_remove_file; </span> </span>// Or like this? </span>use </span>std::io::remove_file; </span></code></pre> This post looks at async overloading in Swift, how we could potentially translate it to Rust, and makes the case that we should make investigating its feasibility a priority for the Async Foundations WG.</p> Choose your calling convention</h2> When designing an API in Rust which performs IO, you have to make a decision whether you want it to be synchronous, asynchronous, or both.</p> If you choose "synchronous" it's likely easier to write, iterate on, and use. But it may be less performant, and interacting with async APIs can be tricky.</p> Choosing "asynchronous" will often mean better performance (especially on multi-core machines), better compatibility with a lot of other networking operations. But it is often significantly harder to author and maintain.</p> If you author both so you can let end-users decide what they want to consume, not only will you incur the cost of writing the same code twice. You also need to choose how to expose both APIs to end-users. This leads to situations like with the mongodb</code> crate</a> which exposes a feature-flagged sync</code></a> submodule which exposes a copy of all IO types for use in synchronous code.</p> There's a clear "primary API" (async), and "secondary API" (sync). From a maintainer's perspective this means that whenever a new async API is introduced, you need to manually ensure that the sync counterpart is used as well. And from a user's perspective you need to make sure you don't accidentally call the wrong API since both live in the same namespace.</p> Another example are the postgres</a> and tokio-postgres</code></a> crates which split the sync/async APIs out into two different crates instead. This solves the namespace problem, but still requires manually ensuring both APIs are updated. And end-users still need to remember to import the right crate for their purpose.</p> The current state of crates isn't unworkable</em>: we know Rust is used in production, often to great success. But I don't think it's hard to argue that it isn't particularly great</em> either. The fact that we need feature flags, submodules, or even separate crates is a direct artifact of the limitations of Rust as a language required to create good designs</em>, and not an issue with any of the ecosystem crates.</p> The standard library</h2> There's no arguing that Async Rust is hot right now. Companies both large and small have made commitments to using Rust, and Async in particular is integral to Rust's adoption. The Async WG (which I'm part of) is looking at integrating parts of the ecosystem into the stdlib; including core traits, but also networking abstractions other IO facilities. And unfortunately the logical endpoint of our current trajectory would be a significant increase in API surface of the stdlib</strong>.</p> As with all things Rust: nothing is certain until it's stabilized. But "async compat" has been a recurring theme, and the Async WG is looking at ways to include runtime-independent async APIs in the stdlib</a>. Which if we were to add an async counterpart for every IO API currently in the stdlib, we'd end up exposing over 300 new types</a> — not accounting for methods.</p> To get a sense of what this would feel like we can look at other language runtimes. Python</a>, C#</a>, and Node.js all have this sync/async split. I'm most familiar with Node.js which has 3 different calling conventions (callbacks, async/await, blocking). Which in practice looks like this:</p> // Delete a file with the default async callback-based API. </span>import </span>{ </span>unlink </span>} </span>from </span>'</span>fs</span>'; </span>unlink</span>('</span>./tmp/foo.txt</span>', (</span>err</span>) </span>=> </span>{ </span> </span>if </span>(</span>err</span>) </span>throw </span>err</span>; </span> </span>console</span>.</span>log</span>('</span>success</span>'); </span>}); </span></code></pre> // Delete a file with the recommended async Promise-based API. </span>import </span>{ </span>unlink </span>} </span>from </span>'</span>fs/promises</span>'; </span>await </span>unlink</span>('</span>./foo.txt</span>'); </span>console</span>.</span>log</span>('</span>success</span>'); </span></code></pre> // Delete a file with the sync blocking API. </span>import </span>{ </span>unlinkSync </span>} </span>from </span>'</span>fs</span>'; </span>unlinkSync</span>('</span>./foo.txt</span>'); </span>console</span>.</span>log</span>('</span>success</span>'); </span></code></pre> Rust's counterpart to Node's fs.unlink</code> is std::fs::remove_file</code></a>. Say we wanted to expose an async counterpart to it as part of the stdlib. We'd have a few options:</p> We could create a new submodule (e.g. std::async_fs</code>) to contain a full counterpart of std::fs</code>, including std::async_fs::remove_file</code>. This approach is similar to how Python supports both import io</code></a> and import asyncio</code></a>.</p> </li> We could add async types directly under std::fs</code>. So we'd have both std::fs::remove_file</code> and std::fs::async_remove_file</code>. This approach is similar to the way Node.js's exposes sync APIs. But having to prefix each call with async_</code> and suffix with .await</code> doesn't make this an appealing option for Rust. Which is likely why I haven't heard anyone who prefers this approach over submodules. Even if prefixing submodules could lead to gems such as std::async_sync</code></a> 1</a></sup>.</p> </li> </ol> ^{1</sup> I don't believe the libs team would seriously let async_sync</code> be a part of the stdlib. I've heard talk about deprecating std::sync</code> and splitting it into separate submodules such as std::lock</code> and std::atomic</code>. Which means we'd likely sooner see a std::async_lock</code> than std::async_sync</code>. But I still find it amusing to think about an alternate future where we'd have sync</code>, async_sync</code>, async</code>, Async</code>, Sink</code>, AsyncSink</code> and Sync</code> in the stdlib.</p> </div> But both options share a core issue: APIs still need to be manually matched to the context they're called in. If you're in a sync context, std::async_fs::remove_file</code> cannot be called. You need to create an async context first.</p> And likewise if you're in an async context: std::fs::remove_file</code> can</em> be used, but it's most likely not what you want: you probably want the async version instead. This means that depending on the type of context you're in, half the IO APIs in the stdlib would be of no use to you. Which leaves the question: Is there a way we could prevent manually needing to match APIs in the stdlib to their right calling convention?</strong></p>}Async overloading in Swift</h2> In an amendment to SE-0296: Async Await</a> made last month, Swift has added support for async overloading which solves this exact problem! It enables sync and async functions of the same name to be declared in the same scope, leaving it up to the caller's context to determine which variant is the right one:</p> // Existing synchronous API </span>func </span>doSomethingElse() { ... } </span> </span>// New and enhanced asynchronous API </span>func </span>doSomethingElse() async { ... } </span></code></pre> Calling async functions from sync functions triggers an error like this:</p> func </span>f() { </span> </span>// Compiler error: `await` is only allowed inside `async` functions </span> await doSomething() </span>} </span></code></pre> But if you're trying to call the sync variant of an overloaded function inside an async function, you'd see the following error:</p> func </span>f() async { </span> </span>// Compiler error: Expression is 'async' but is not marked with 'await' </span> doSomething() </span>} </span></code></pre> However it's still possible to call the synchronous version of the API by creating a synchronous context inside the async block:</p> func </span>f() async { </span> </span>let</span> f2 = { </span> </span>// In a synchronous context, the non-async overload is preferred: </span> doSomething() </span> } </span> f2() </span>} </span></code></pre> But async overloading is not required. It's still possible to declare async functions without overloading: they'll just only work in async contexts. And similarly it's possible to declare sync functions which will work in all contexts.</p> Unlike other languages, Swift will not need to effectively double the API surface of their stdlib to support their newly added async functionality. From a user's perspective the same API will "work as expected" in both sync and async contexts. And that is something Rust can learn from.</p> Bringing async overloading to Rust</h2> Now that you know how Swift has solved this issue, it asks the question: Could we integrate a system like this into Rust. The fact that I'm talking about this at all probably gives away that I think we can. But what would that look like?</p> For async functions, overloading could be easy enough. Adapting the Swift example we saw earlier, the following should be able to work:</p> // Existing synchronous API </span>fn </span>do_something_else</span>() { ... } </span> </span>// New and enhanced asynchronous API </span>async </span>fn </span>do_something_else</span>() { ... } </span></code></pre> We could even give a specialization spin</a> to the API, and make the syntax something like:</p> default </span>fn </span>do_something_else</span>() { ... } </span>async </span>fn </span>do_something_else</span>() { ... } </span></code></pre> But I don't know what the right syntax for this would be. Maybe it would warrant a different approach altogether! Today, if we try awaiting an async function inside a synchronous closure, we're met with the following diagnostic:</p> error[E0728]: `await` is only allowed inside `async` functions and blocks </span> --> src/lib.rs:4:5 </span> \| </span>3 \| fn f() { </span> \| - this is not `async` </span>4 \| do_something().await; </span> \| ^^^^^^^^^^^^^^^^^^^^ only allowed inside `async` functions and blocks </span></code></pre> With the overload added, we can start suggesting fixes for errors like this too 2</a></sup>. For example:</p> ^2</sup>As an aside; I'm super excited for the upcoming diagnostics changes</a>. The diff view is looking really good! It's just a bummer my blog theme doesn't allow me to custom-color diagnostics output yet. For now imagine the diagnostics in my blog have pretty colors like the compiler does, heh.</p> </div> \| </span>help: try removing `.await` from the function call </span> \| </span>3 \| async fn f() { </span>4 - do_something().await; </span>4 + do_something(); </span> \| </span></code></pre> And calling the overloaded sync function from an async context could produce a diagnostic like this:</p> warning: unused implementer of `Future` that must be used </span> --> src/lib.rs:4:5 </span> \| </span>3 \| async fn f() { </span>4 \| do_something(); </span> \| ^^^^^^^^^^^^^^^ </span> \| </span> = note: this function is called in an async context and uses the async overload </span> = note: `#[warn(unused_must_use)]` on by default </span> = note: futures do nothing unless you `.await` or poll them </span> \| </span>help: try adding `.await` to the function call </span> \| </span>3 \| async fn f() { </span>4 \| do_something().await; </span> \| ++++++ </span></code></pre> Just like in Swift, using the synchronous overload in an async context could be done by writing:</p> async </span>fn </span>f</span>() { </span> </span>let </span>f2 </span>= \|\| { </span> doSomething() </span> }; </span> </span>f2</span>() </span>} </span></code></pre> For traits we could imagine something similar as for functions. Supposing the default</code> keyword (specialization) could also be used for overloads; we could imagine we could write something like this for an async version of Read</code></a>:</p> /// The synchronous `Read` trait </span>pub</span> default </span>trait </span>Read { </span> </span>fn </span>read</span>(&</span>mut </span>self</span>, </span>buf</span>: &</span>mut</span> [</span>u8</span>]) -> Result<</span>usize</span>>; </span>} </span> </span>/// The asynchronous `Read` trait </span>pub</span> async </span>trait </span>Read { </span> async </span>fn </span>read</span>(&</span>mut </span>self</span>, </span>buf</span>: &</span>mut</span> [</span>u8</span>]) -> Result<</span>usize</span>>; </span>} </span></code></pre> Depending on how the traits are implemented, users could end up implementing none, either, or both. Which variants of the trait are implemented could then be highlighted through the docs, for example:</p> --------------------- </span>Trait Implementations </span>--------------------- </span>Debug </span>FromRawFd </span>Read </span>Read (async) </span>Write </span>Write (async) </span></code></pre> This really is all a rough sketch. I don't know whether using default</code> as a keyword would make sense here at all, or what the right way of highlighting this in the docs are. What I hope to achieve with this is to share a glimpse of how this could work end-to-end for Rust users. Docs, diagnostics, writing, and consuming code are all different ways to experience the same feature. And I hope this provides an idea of what an integration could possibly look like for both library consumers and authors alike.</p> Potential caveats</h2> Backwards incompatibility</h4> As far as I can tell Rust doesn't have anything yet that would preclude us from going down this path. Indeed adding this would cause a delay in "shipping" async Rust. But it would make it so we don't need to effectively double the surface area of the stdlib, which to me is worth it (this is coming from someone who's helped author an async copy of virtually every stdlib API).</p> If we decide this is worth exploring, the one thing we shouldn't do is stabilize std::stream</code> before our explorations are over. I don't believe any other APIs are included yet in nightly which could pose an issue for async overloading, so as long as we don't stabilize that while we figure this out, we should be good! 3</a></sup>.</p> ^{3</sup> And again, this is coming from someone who helped author the Stream</code> RFC, and added Stream</code> to nightly. Please believe me that if someone wants to stabilize async iterators, it's me. But I think async overloading is the kind of change we should consider and definitively rule out before proceeding we attempt to stabilize the Stream</code> API as-is.</p> </div>}Overloading existing stdlib functions</h4> One issue to be aware of is that unlike Swift we cannot immediately fail if a synchronous overload is selected in an async function. Rust's async models allows for delayed .await</code>ing, which means we cannot error at the call-site. Instead we'll likely need to hook into the machinery that enables #[must_use]</code>; allowing us to validate whether the returned future is actually awaited — and warn if it's not. Even though this is slightly different from Swift appears to do things, it should not present an insurmountable hurdle.</p> Feasibility of async parity</h4> One of the goals of the async-std</a> project was to gauge the feasibility of creating a dual of the stdlib's IO APIs using async Rust. And two years in we can conclude it's been an overwhelming success: the only stdlib APIs we haven't been able to faithfully reimplement in async Rust are those which require async closures and async traits 4</a></sup>. But now that the Async Foundations WG is adding those capabilities to the language, there are no blockers remaining for API parity between sync and async Rust.</p> ^4</sup>An example of an API which requires async traits is async-std</code>'s Stream::collect</code></a>. We've managed to work around it, but had to box some items. An example of an API which requires async closures is Stream::filter</code></a>. The callback provided to the method is not async right now because there's no way to express the lifetime of the borrowed item. But once we have async closures this will become possible to express. Pretty much all differences between std and async-std are minor issues like these, which we're fairly certain can be solved once Async Rust gains more capabilities.</p> </div> Past async overloading: code reuse</h4> Finally, there's an interesting question to ask whether it's possible to reuse code past an async overload. You could imagine a sync and an async parser containing identical logic, except for the internal use of .await</code>. To which degree does Swift's model limit code reuse? That should be something the async WG should answer once it decides async overloading is indeed desireable, and starts looking at ways it could be implemented.</p> Returning impl Future</code> instead of async fn</code></h4> Without TAIT</a>s async fn</code>s cannot be named, and thus for the stdlib it'll be desireable to return manual futures from APIs instead. This means that we might want to write something like this:</p> default </span>fn </span>do_something_else</span>() { ... } </span>fn </span>do_something_else</span>() -> DoSomethingElse { ... } </span> </span>struct </span>DoSomethingElse; </span>impl </span>Future </span>for </span>DoSomethingElse { ... } </span></code></pre> Luckily the compiler is aware of which traits are implemented on types, and could validate that this indeed resolves to a valid async overload. We should probably even make it so -> impl Future<Output = io::Result<()>></code> works, but -> io::Result<impl Future<Output = ()>></code> does not. This would likely be quicker to validate, and keep things feeling like "async fn overloading" only.</p> Overloads for other effects</h4> A question which has come up is whether this mechanism could be used for other effects as well. For example, it could be conceivable that in a try fn</code> we could expect to call a fallible API, while in a non-try fn</code> we'd expect the infallible method:</p> pub</span> default </span>fn </span>reserve</span>(&</span>mut </span>self</span>, </span>additional</span>: </span>usize</span>); </span>pub</span> try </span>fn </span>reserve</span>(&</span>mut </span>self</span>, </span>additional</span>: </span>usize</span>) -> throws ReserveError; </span></code></pre> In my Fallible Iterator Adapter</a> post I pointed out that we're missing a fair number of fallible variants of existing APIs. Boats has since authored The Problem of Effects in Rust</a> which covers the problem from a language perspective, and shows how if we introduced a different kind of closure we could eliminate the need for try_</code> iterator adapters.</p> RFC 2116: Fallible Collection Allocation</a> covers adding try_</code> variants of the base building blocks. But the implementation has decided not to add try_</code> variants for all allocation APIs since that would increase the API surface too much. Which is a similar reasoning given for why we don't have fallible variants for all iterator APIs either.</p> The issues we're covering in this post, Boats' effect post, and the issues in RFC 2116 all seem related to each other. I don't know if "fallible overloading" could provide a partial way out of it, but maybe there's something to it? The possibilities different kinds of overloading could bring certainly are interesting, and probably worth thinking about more.</p> Other</h4> There are two topics which I haven't thought about, but should be thought about by the working group:</p> How does async overloading interact with dyn Trait</code>?</li> How does async overloading interact with FFI?</li> </ol> Basing on what I've seen come up in conversations there might be ways to make this work, but it should be looked at more closely as part of a feasibility probe.</p> Conclusion</h2> Currently the async foundations roadmap does include a note on "async overloading"</a>, but it's marked as "slightly past the horizon". I think given this has implications for async iteration, async IO, and every other async libs API, we should make sure we've properly investigated async overloading before we make any changes to the stdlib which would be incompatible with it.</strong></p> To summarize what we've covered so far:</p> Async Rust is currently on a trajectory to doubling the API surface of IO APIs in the stdlib.</li> Async overloading could provide a way out for us, enabling us to add async IO APIs to the stdlib while keeping the number of IO APIs in the stdlib to its current number.</li> As long as we don't stabilize Stream</code>, async IO traits, or any other async IO types we should be capable of introducing async overloading without any sharp edges.</li> Because async overloading informs the implementation of all of the async types in the stdlib, investigating its feasiblity should be a priority for the async WG.</li> </ol> My hopes of this post is that it convinces folks that async overloading is an idea worth taking seriously before we continue with any other designs. And to perform a feasibility study whether this could actually work from a language, libs, and compiler perspective.</p> Thanks to: Eric Holk, Irina Shestak, James Halliday, Scott McMurray, and Wesley Wiser for helping review this post prior to publishing.</em></p> optimizing hashmaps even more Sat, 08 May 2021 00:00:00 +0000 In our last post we took a look at possible ways we could improve the ergonomics of Rust's refcounting APIs. In this post we'll be looking at Hashmap: how it's currently implemented, how we could optimize it further, and finally directions we could explore to enable the compiler to automatically apply these optimizations.</p> Disclaimer:</strong> I'm not an expert. I'm not on any team involved with const execution, nor do I hold any decision making power. This exists to share ideas and curiosities of what might be possible in Rust in the spirit of sharing posts with pals.</p> Hashmaps and hashing algorithms</h2> In 2017 Google introduced a new kind of hashmap for C++: the Swisstable hashmap</a>. This structure uses SIMD instructions to compute hashes, which is a great match for modern computer hardware. So any improvements to the hashing algorithm would have an outsize impact on performance, and in turn resource efficiency.</p> In the cppcon 2017 talk on Swisstable</a>, the authors mention that across all of Google's code, they found a non-trivial amount of time was spent computing hashes 1</a></sup>. Amanieu</a> from the libs team created a Rust variant of the same algorithm for the Hashbrown</a> crate, which has since become the default Hashmap implemention for Rust.</p> ^{1</sup> 1% of CPU and 4% of RAM of all of Google's C++ code</em> is spent doing things inside hashtables. Considering the scale of Google's deployment, the amount of CPU cycles spent is enormous</em>. And it doesn't even consider software running on user devices such as Android and Chrome, or programming languages other than C++.</p> </div>}However we've known for a long time about a faster hashmap than Swisstable: the perfect hashtable</a>. The idea is that if you know all your keys ahead of time, you can create a lookup table which never creates collisions, never has to re-index the hashmap, or grow in size2</a></sup>. Though it's not always possible to know all keys ahead of time, so there are limits to when it can be used.</p> ^2</sup>There are a lot of considerations and tradeoffs for hashing algorithms. Sy Brand gave a brief overview on Twitter</a> in reply to this post.</p> </div> Even so, let's take a look at when it does make sense to use perfect hashtables, and show examples of how to implement them.</p> enums as keys</h2> Imagine we're building a protocol. The protocol has support for "metadata" and "body data". The metadata consists of keys and values. We could imagine its use could look something like this:</p> use </span>std::collections::HashMap; </span> </span>let mut</span> metadata: HashMap<String, String> = HashMap::new(); </span> </span>metadata.</span>insert</span>("</span>content_type</span>".</span>into</span>(), "</span>fast_proto</span>".</span>into</span>()); </span>metadata.</span>insert</span>("</span>encryption_key</span>".</span>into</span>(), generator.</span>next_key</span>()); </span>metadata.</span>insert</span>("</span>content_fingerprint</span>".</span>into</span>(), </span>bodyfingerprint</span>()); </span>metadata.</span>insert</span>("</span>content_length</span>".</span>into</span>(), </span>bodylength</span>().</span>into</span>()); </span> </span>request.</span>send_metadata</span>(metadata).await?; </span></code></pre> We create a hashmap, insert several key-value pairs into it, and finally serialize it over the wire. Now imagine these are the only</em> keys that are valid to insert into the hashmap. Since we know all possible representations up front, we could capture them within an enum:</p> use </span>std::collections::HashMap; </span> </span>enum </span>KeyName { </span> ContentType, </span> EncryptionKey, </span> ContentFingerprint, </span> ContentLength, </span>} </span>impl </span>Display </span>for </span>KeyName { ... } </span> </span>let mut</span> metadata: HashMap<KeyName, String> = HashMap::new(); </span> </span>metadata.</span>insert</span>(KeyName::ContentType, "</span>fast_proto</span>".</span>into</span>()); </span>metadata.</span>insert</span>(KeyName::EncryptionKey, generator.</span>next_key</span>()); </span>metadata.</span>insert</span>(KeyName::ContentFingerprint, body.</span>fingerprint</span>()); </span>metadata.</span>insert</span>(KeyName::ContentLength, body.</span>length</span>().</span>into</span>()); </span> </span>request.</span>send_metadata</span>(metadata).await?; </span></code></pre> Because we're using an enum we know all of our keys ahead of time 3</a></sup>, and can simply match</code> on the keys. Yet the code above will still expand to roughly 1200 lines of assembly</a>. This is because the Rust compiler currently isn't clever enough to lower the hashmap into a lookup table.</p> ^{3</sup> Yes yes, #[non_exhaustive]</code> would change that. But we aren't using that here.</p> </div>}Now what would happen if we had a "sufficiently smart compiler". The compiler would be able to lower the hashmap to a fixed-sized array containing Option<String></code> which uses match</code> to access the data within 4</a></sup>:</p> ^{4</sup> We may not always want to use an array to store the data. But we're dealing with a small key size here, so it's fine</em>.</p> </div>}pub struct </span>HashMap { </span> </span>inner</span>: [Option<String>; 4], </span>} </span> </span>impl </span>HashMap { </span> </span>pub fn </span>insert</span>(&</span>mut </span>self</span>, </span>key</span>: KeyName, </span>value</span>: String) -> Option<String> { </span> </span>match</span> key { </span> KeyName::ContentType => </span>self</span>.inner[</span>0</span>].</span>replace</span>(value), </span> KeyName::EncryptionKey => </span>self</span>.inner[</span>1</span>].</span>replace</span>(value), </span> KeyName::Fingerprint => </span>self</span>.inner[</span>2</span>].</span>replace</span>(value), </span> KeyName::ContentLength => </span>self</span>.inner[</span>3</span>].</span>replace</span>(value), </span> } </span> } </span> </span> </span>pub fn </span>get</span>(&</span>self</span>, </span>key</span>: KeyName) -> Option<&String> { </span> </span>match</span> key { </span> KeyName::ContentType => </span>self</span>.inner[</span>0</span>].</span>as_ref</span>(), </span> KeyName::EncryptionKey => </span>self</span>.inner[</span>1</span>].</span>as_ref</span>(), </span> KeyName::Fingerprint => </span>self</span>.inner[</span>2</span>].</span>as_ref</span>(), </span> KeyName::ContentLength => </span>self</span>.inner[</span>3</span>].</span>as_ref</span>(), </span> } </span> } </span>} </span></code></pre> What we're seeing here is that we've replaced the hashing function with a match</code> statement. All data is stored within a fixed-sized array of Option</code>s, and each enum variant directly maps to one of the slots within array. This removes the overhead of hashing entirely, and opens the door for further optimizations (such as inlining).</p> Some might argue that: "We could just write this code ourselves", but writing custom data structures has costs associated with it: we need to realize we need the structure in the first place, then write the structure, maintain it, and teach everyone else working on the codebase about it. Even though it might be faster than Rust's default hashmap, going down this route has extra costs.</p> static strings as keys</h2> In many ways our example of keying into a hashmap using an enum is a bit too</em> perfect. It assumes the author has awareness that their data is in fact enumerable, and willingness to refactor the set of all keys into an enum. But it's also risky: what if want to add more keys later. What if those keys may not be known during compilation? I personally often prefer to keep things simple when I'm not sure how things might change in the future.</p> Code in the real world is often far less pristine. When trying to deliver a feature under time pressure we're often more focused on getting it working and tested, than on performance. After all it's usually fast enough</em> anyway. We'll often key into hashmaps using strings, which have to be inlined into the created binary. The assembly output for those inlined strings will look somewhat like this:</p> .L__unnamed_1: </span> .ascii "</span>content_type</span>" </span> </span>.L__unnamed_2: </span> .ascii "</span>encryption_key</span>" </span> </span>.L__unnamed_3: </span> .ascii "</span>content_fingerprint</span>" </span> </span>.L__unnamed_4: </span> .ascii "</span>content_length</span>" </span></code></pre> A compiler not only knows which strings are being inlined. It also knows which strings are used as inputs to which methods. This is to say: a compiler is very much capable of capturing the full set of inputs during compilation 5</a></sup>, which is provides us with data not unlike a Rust enum. That means at least theoretically it should be possible to generate a perfect hashmap for the following input:</p> ^{5</sup> Not sure if this currently occurs in LLVM or not. But I know I'm able to obtain this data through Salsa in Rust-Analyzer.</p> </div>}use </span>std::collections::HashMap; </span> </span>let mut</span> metadata: HashMap<&</span>'static str</span>, String> = HashMap::new(); </span> </span>metadata.</span>insert</span>("</span>content_type</span>", "</span>fast_proto</span>".</span>into</span>()); </span>metadata.</span>insert</span>("</span>encryption_key</span>", generator.</span>next_key</span>()); </span>metadata.</span>insert</span>("</span>content_fingerprint</span>", body.</span>fingerprint</span>()); </span>metadata.</span>insert</span>("</span>content_length</span>", body.</span>length</span>().</span>into</span>()); </span> </span>request.</span>send_metadata</span>(metadata).await?; </span></code></pre> And the generated perfect hashmap would be:</p> pub struct </span>HashMap { </span> </span>inner</span>: [Option<String>; 4], </span>} </span> </span>impl </span>HashMap { </span> </span>pub fn </span>insert</span>(&</span>mut </span>self</span>, </span>key</span>: &</span>str</span>, </span>value</span>: String) -> Option<String> { </span> </span>match</span> key { </span> "</span>content_type</span>" => </span>self</span>.inner[</span>0</span>].</span>replace</span>(value), </span> "</span>encryption_key</span>" => </span>self</span>.inner[</span>1</span>].</span>replace</span>(value), </span> "</span>content_fingerprint</span>" => </span>self</span>.inner[</span>2</span>].</span>replace</span>(value), </span> "</span>content_length</span>" => </span>self</span>.inner[</span>3</span>].</span>replace</span>(value), </span> _ => </span>unsafe </span>{ std::hint::unreachable_unchecked() }, </span> } </span> } </span> </span> </span>pub fn </span>get</span>(&</span>self</span>, </span>key</span>: &</span>str</span>) -> Option<&String> { </span> </span>match</span> key { </span> "</span>content_type</span>" => </span>self</span>.inner[</span>0</span>].</span>as_ref</span>(), </span> "</span>encryption_key</span>" => </span>self</span>.inner[</span>1</span>].</span>as_ref</span>(), </span> "</span>content_fingerprint</span>" => </span>self</span>.inner[</span>2</span>].</span>as_ref</span>(), </span> "</span>content_length</span>" => </span>self</span>.inner[</span>3</span>].</span>as_ref</span>(), </span> _ => </span>unsafe </span>{ std::hint::unreachable_unchecked() }, </span> } </span> } </span>} </span></code></pre> hybrid static + dynamic keys</h2> If our keys are only ever known during runtime, we know the best hashmap we can use is Hashbrown</code>. But what if some of our keys are known during compilation?</p> This is where we can use a "hybrid" hashmap 6</a></sup>: we generate a lookup table for the known keys, and use a regular hashmap as a fallback for all other keys. This approach is used by hyperium/http</code></a>'s HeaderMap</code> struct. It keeps track of commonly used HTTP headers in a lookup table, and falls back to a hashmap for all other headers.</p> ^{6</sup> I made up the term "hybrid hashmap" for the purpose of this post. Let me know if you know of a term used in literature to describe this same structure.</p> </div> Here's a usage example of a hashmap which uses both dynamic and static data:</p>}use </span>std::collections::HashMap; </span> </span>let mut</span> metadata: HashMap<String, String> = HashMap::new(); </span> </span>metadata.</span>insert</span>("</span>content_type</span>".</span>into</span>(), "</span>fast_proto</span>".</span>into</span>()); </span>metadata.</span>insert</span>("</span>encryption_key</span>".</span>into</span>(), generator.</span>next_key</span>()); </span>metadata.</span>insert</span>(</span>gen_keyname</span>(), </span>gen_headername</span>()); </span>// this generates values at runtime </span> </span>request.</span>send_metadata</span>(metadata).await?; </span></code></pre> For this code we're able to generate the following implementation. We create a lookup table for our known keys, and fall back to a hashmap for all other keys:</p> pub struct </span>HashMap { </span> </span>inner</span>: [Option<String>; 4], </span> </span>map</span>: std::collections::HashMap<String, String>, </span>} </span> </span>impl </span>HashMap { </span> </span>pub fn </span>insert</span>(&</span>mut </span>self</span>, </span>key</span>: String, </span>value</span>: String) -> Option<String> { </span> </span>match</span> key.</span>as_ref</span>() { </span> "</span>content_type</span>" => </span>self</span>.inner[</span>0</span>].</span>replace</span>(value), </span> "</span>encryption_key</span>" => </span>self</span>.inner[</span>1</span>].</span>replace</span>(value), </span> "</span>content_fingerprint</span>" => </span>self</span>.inner[</span>2</span>].</span>replace</span>(value), </span> "</span>content_length</span>" => </span>self</span>.inner[</span>3</span>].</span>replace</span>(value), </span> _ => </span>self</span>.map.</span>insert</span>(key, value), </span> } </span> } </span> </span> </span>pub fn </span>get</span>(&</span>self</span>, </span>key</span>: &</span>str</span>) -> Option<&String> { </span> </span>match</span> key { </span> "</span>content_type</span>" => </span>self</span>.inner[</span>0</span>].</span>as_ref</span>(), </span> "</span>encryption_key</span>" => </span>self</span>.inner[</span>1</span>].</span>as_ref</span>(), </span> "</span>content_fingerprint</span>" => </span>self</span>.inner[</span>2</span>].</span>as_ref</span>(), </span> "</span>content_length</span>" => </span>self</span>.inner[</span>3</span>].</span>as_ref</span>(), </span> _ => </span>self</span>.map.</span>get</span>(key), </span> } </span> } </span>} </span></code></pre> A nice property about this approach is that if we only ever use known keys, we'll get speeds similar to that of a lookup table. It's only once we start using custom keys that we branch into a slower path and start allocating for the hashmap.</p> Looking ahead</h2> While it's possible for humans to look for instances where a hashmap can be converted into a lookup table, for Rust only a compiler will be able to catch all</em> cases. This is because virtually no code in Rust is ever completely self-contained.</p> Say we have a hashmap defined in crate A. And we have a crate B which uses that hashmap. If B only ever uses A with static keys we're able to optimize for that use. However crate A will cannot know ahead of time what the usage patterns of B will be like.</p> Add in more crates, transient wrappers, and legacy code - and it quickly becomes clear why it's ideal to let machines handle these kinds of optimizations. However the question then becomes: how would we enable this in Rust?</p> I think if we had access to the following information during const / specialization we'd be able to create these kinds of optimizations:</p> Provide a way to enumerate all static inputs for a function.</li> Provide a way to tell if there are runtime inputs for a function.</li> Provide a way to count the number of invocations a function has within the codebase.</li> </ol> All of this information can be known by programmers ahead of time by reading the code they're working on. This means that a compiler can know this information too. The question really is how (and where) to expose this information.</p> My gut feeling is that we should be able to figure this out somewhere during const evaluation and specialization 7</a></sup> 2</a></sup>. Using an entirely made-up, fantasy pseudo version of Rust I'd imagine we'd be able to create a new hashmap using something like this:</p> ^{7</sup> Yes I know that's not a separate step, - less of a time, more of a place I guess.</p> </div>}^{2</sup> I don't believe we can just specialize every hashmap + hasher combo. I'm thinking we might be able to specialize Rust's default hashmap though. As I understand it we don't guarantee much of anything other than it being fast and not being a DDoS vector.</p> </div>}#[</span>this_is_a_specialization</span>] </span>const fn </span>create_hashmap</span><K: </span>'static</span>, V>(</span>m</span>: Map<K, V>) -> HashMap { </span> </span>// how often is this map's `get` function called? </span> </span>if </span>mem::invocation_count_of(m.get) == </span>0 </span>{ </span> </span>// return stub structure because we know the data is never accessed </span> } </span> </span> </span>// iterate over all static inputs provided to m.insert </span> </span>for </span>(k, _) in mem::static_inputs_of(m.insert) { </span> </span>// iterate over all keys and return a lookup table </span> } </span> </span> </span>// count how often m.insert is called with non-static input </span> </span>if </span>mem::dynamic_input_count_of(m.insert) != </span>0 </span>{ </span> </span>// if there are runtime (non-static) arguments we want to return a hybrid hashmap </span> } </span>else </span>{ </span> </span>// yay, return a lookup table </span> } </span>} </span></code></pre> Maybe there are other proposals in progress for const execution / specialization which cover these cases as well. Admittedly adding a bunch of new APIs backed by / compiler built-in /</code> is a bit of a cop-out, and I'd love to learn if there could be ways to do the same which more closely integrate with what we already have.</p> I hope this paints a rough picture of what I'm thinking. I wish I knew more about the present and future of Rust's const engine to design a better API. But my goal with this post is less to propose something concrete, and more to convince the right people this is something worth exploring. I hope in that regard I'm doing alright so far (:</p> Conclusion</h2> I hope everyone walks away from this post with two things: an understanding of how to create a lookup table/perfect hashmap in Rust, and excitement about a future where our compilers can apply these optimizations for us.</p> The idea of transparently optimizing existing patterns based on usage is fascinating to me. Usually when I think about these kinds of optimizations it's things like compilers optimizing math by converting it to bitwise instructions. But imagine how cool it would be if we could optimize entire data structures based on how they're used. This type of optimization doesn't need to stop with hashmaps either: I got interested in this topic because I think we can optimize channels significantly if we can inspect their use during compilation 8</a></sup>.</p> ^{8</sup> For example: we could optimize an MPMC channel into a one-shot SPSC channel if we know both the sender and receiver are never cloned, and only one item is ever sent.</p> </div> Another thing which excites me about this is that if we're able to lower a hashmap into a match</code> statement, we can proceed to apply all optimizations we use to match statements. This means optimizations such as removing unused enum variants, reordering keys for quicker lookups, and even lowering the match statement itself to more efficient structures such as prefix-tries. Rust's match</code> performance is probably not where it could be yet either, and the more we can express code of one kind in terms of another, the more impactful optimizations can become.</p> Thanks for reading!</p>} refcounts Fri, 07 May 2021 00:00:00 +0000 MyData I've wondered about for a while now is how to improve the ergonomics of some of the structures in std::sync</code>. There's quite a bit of excitement gathering about ghost-cell</code></a> (and the paper</a>), and I figured this would be as good a time as any to share some thoughts (questions?) on ergonomics.</p> Like my last post, the intent of this post is less to provide a solution or result of research, but instead share my current thinking on the topic. Rather than asking questions in private, I figured I'd ask questions on here instead (:</p> std::sync::Barrier</code></h2> Let's use std::sync::Barrier</code></a> to center our examples around. This is a data structure which enables synchronization between any number of threads. One example of how you might want to use this is in an HTTP server. You may want to create a connection to a database, connect to a cache server, connect to perhaps some other backend, and only when all of those have succeeded do you proceed to accept connections. If one of them takes a while to initialize (or even fails entirely), our end-users won't be impacted because none of their requests are being handled yet.</p> This structure enables multiple threads to synchronize</em> before starting work. Its use usually looks something like this:</p> use </span>std::sync::{Arc, Barrier}; </span>use </span>std::thread; </span> </span>let mut</span> handles = Vec::with_capacity(</span>10</span>); </span>let</span> barrier = Arc::new(Barrier::new(</span>10</span>)); </span>for </span>_ in </span>0</span>..</span>10 </span>{ </span> </span>let</span> c = Arc::clone(&barrier); </span> </span>// The same messages will be printed together. </span> </span>// You will NOT see any interleaving. </span> handles.</span>push</span>(thread::spawn(</span>move</span>\|\| { </span> println!("</span>before wait</span>"); </span> c.</span>wait</span>(); </span> println!("</span>after wait</span>"); </span> })); </span>} </span>// Wait for other threads to finish. </span>for</span> handle in handles { </span> handle.</span>join</span>().</span>unwrap</span>(); </span>} </span></code></pre> We create an instance of Barrier</code> by passing it a count, wrap it in an Arc</code> (so it can be cloned), and then pass an instance of it to each thread. Important to note is that the number of threads we spawn needs to exactly</em> line up to the number we've apssed into Barrier::new</code>. If the number is too high our application will lock up and never continue. If the number is too low, the application will start before all threads are ready, defeating the point of using the structure in the first place.</p> dynamic Barrier</h2> One difficulty with std::sync::Barrier</code> is that we need to statically</em> pass the number in. This means in order to create the synchronization structure we need to know up front how many items we're trying to synchronize. Say we're dealing with an iterator of unknown length, and we want to exhaust all items before continuing? We don't know the number up front, so we can't use the structure:</p> use </span>std::thread; </span>use </span>std::sync::{Arc, Barrier}; </span> </span>fn </span>from_iter</span>(</span>iter</span>: impl Iterator<Item = MyData>) { </span> </span>let mut</span> handles = Vec::new(); </span> </span>let</span> barrier = Arc::new(Barrier::new(</span>/ ??? /</span>)); </span> </span>for</span> item in iter { </span> </span>// ... create a thread and wait within </span> } </span> </span>// ... join all handles </span>} </span></code></pre> Counting at runtime is not something exceptional though. In our example Arc</code> works entirely at runtime, making work without passing a count ahead of time. Though we don't have a dynamic version of Barrier</code> in the stdlib, we do</em> have one in the ecosystem in the form of: crossbeam::sync::WaitGroup</code></a>. With it we can rewrite our previous example like so:</p> use </span>std::thread; </span>use </span>crossbeam::sync::WaitGroup; </span> </span>fn </span>from_iter</span>(</span>iter</span>: impl Iterator<Item = MyData>) { </span> </span>let mut</span> handles = Vec::new(); </span> </span>let</span> wg = WaitGroup::new(); </span>// 1. create a new instance. </span> </span>for</span> item in iter { </span> </span>let</span> wg = wg.</span>clone</span>(); </span>// 2. clone it </span> handles.</span>push</span>(thread::spawn(</span>move</span>\|\| { </span> println!("</span>before wait</span>"); </span> wg.</span>wait</span>(); </span>// 3. wait to sync </span> println!("</span>after wait</span>"); </span> })); </span> } </span> </span> wg.</span>wait</span>(); </span>// 4. also sync our original wg instance </span> </span> </span>// ... join all handles </span>} </span></code></pre> The upside of using WaitGroup</code> over Barrier</code> is that it's much easier to use, and more widely applicable. The downside is that it's (marginally) less efficient since we need to count the instances at runtime rather than having that information up front 1</a></sup>.</p> ^{1</sup> One could argue that manually passing the count up front can be an extra safeguard - but there's inherent risks to mis-counting the number of instances. So I'm not sure that's much of an extra safeguard. More on that later.</p> </div> I don't want to argue for including WaitGroup</code> in the stdlib since it's unclear how common the use of Barrier</code> is. But I hope to have illustrated that Barrier</code> has limitations in how it can be used to create cross-thread synchronization points.</p>}const Barrier</h2> As we've seen in the first example there's another issue: we need to ensure that the number passed to Barrier::new</code> matches the number of threads we spawn. One way we could fix this is by creating a variable, and passing that same variable to both the loop and Barrier::new</code>:</p> let</span> count = </span>10</span>; </span>let mut</span> handles = Vec::new(); </span>let</span> barrier = Arc::new(Barrier::new(count)); </span>for </span>_ in </span>0</span>..count { </span> </span>let</span> c = Arc::clone(&barrier); </span> </span>/ ... / </span>} </span></code></pre> This definitely works, but it still manually requires synchronizing the value in Barrier</code> and the value in the loop. Even though it's more efficient than using WaitGroup</code>, it's less ergonomic to use. However if we, as humans, look at the code it's immediately clear what the value in Barrier::new</code> should be. Namely: we're calling Arc::clone</code> ten times, so we know ahead of time</em> that the value we pass to Barrier::new</code> should be 10</code>.</p> Now here's what I wonder (and honestly, am not sure about). Given the number of clones can be statically inferred by humans by looking at the code, could we find a way to for this information to become statically available during const</code>?</p> let</span> count = </span>10</span>; </span>let mut</span> handles = Vec::new(); </span>let</span> barrier = Barrier::new() </span>for </span>_ in </span>0</span>..count { </span> </span>let</span> c = barrier.</span>clone</span>(); </span> </span>/ ... / </span>} </span></code></pre> That would allow us to create a dynamic barrier and compile-time barrier with similar APIs, but different assurances. The dynamic barrier would be more flexible and work for unknown inputs, and the compile-time barrier would be more efficient, but only work if the number of items is known at compile-time.</p> Conclusion</h2> Using std::sync::Barrier</code> only acts an example. The same reasoning could be applied to e.g. Arc</code>, Rc</code>, channel</code>, and more. For example for channels it could be interesting if we could compute ahead of time what the max number of items that might be in flight are, and fill that in as the queue size. Caveats and circumstances apply, but the general premise is: "If some item count is known at compile time, is there a way we can pass that number back to our structures?"</em></p> Crates such as static_rc</code></a> are getting close to this idea, but still require manually passing in the count. I'm wondering whether it could be possible to eliminate that count entirely.</p> edit</strong>: I found this thread</a> by the author of static_rc</code> which asks very similar questions as we do here. Also lcnr shared some thoughts on Twitter about the feasibility</a> of what refcounting in const.</p> Before closing out this post, I want to re-iterate once more that nothing of this is a concrete proposal. I'm not even advocating for including crossbeam::sync::WaitGroup</code> in the stdlib. The purpose of this post is to share thinking, and ask questions about what is desireable and possible within the (future of) the Rust language.</p> And with that, thanks so much for reading this - happy weekend!</p> Futures Concurrency Notes: join and const-eval Sat, 16 Jan 2021 00:00:00 +0000 Happy new year everyone! Today we're trying something new: less of a blog post, more of research notes</em>. This is less of a "here's something I've concluded", and more of: "here's something I'm thinking about". Today's topic is: "How can we add Future::{try_}join</code> and {try_}join!</code> to the stdlib in a way that feels consistent?"</p> What does joining Futures do?</h2> A Future in Rust is best though of as a "value which eventually becomes available". It's not specified when</em> a value becomes available, so using .await</code> allows us to wait for it until it's available.</p> Sometimes we want to wait on more than one future at the time: after all, when we're waiting on things, we can do other things in the mean time. And one way to do this is by calling join</code>.</p> async-std</code> exposes a Future::join</code> method, and async-macros</code> exposes a join!</code> macro. An example joining two futures:</p> let</span> a = future::ready(</span>1</span>u8</span>); </span>let</span> b = future::ready(</span>2</span>u8</span>); </span>assert_eq!(join!(a, b).await, (</span>1</span>, </span>2</span>)); </span> </span>let</span> a = future::ready(</span>1</span>u8</span>); </span>let</span> b = future::ready(</span>2</span>u8</span>); </span>assert_eq!(a.</span>join</span>(b).await, (</span>1</span>, </span>2</span>)); </span></code></pre> However once we start joining more than two futures, the output types become different:</p> let</span> a = future::ready(</span>1</span>u8</span>); </span>let</span> b = future::ready(</span>2</span>u8</span>); </span>let</span> c = future::ready(</span>3</span>u8</span>); </span>assert_eq!(join!(a, b, c).await, (</span>1</span>, </span>2</span>, </span>3</span>)); </span> </span>let</span> a = future::ready(</span>1</span>u8</span>); </span>let</span> b = future::ready(</span>2</span>u8</span>); </span>let</span> c = future::ready(</span>3</span>u8</span>); </span>assert_eq!(a.</span>join</span>(b).</span>join</span>(c).await, (</span>1</span>, (</span>2</span>, </span>3</span>))); </span></code></pre> As you can see, each invocation of Future::join</code> returns a tuple. But that means that chaining calls to it starts to nest tuples, which becomes hard to use. And it becomes more nested the more times you chain. Oh no!</p> In contrast, the join!</code> macro dynamically grows the number of items returned in the tuple. This is possible because macros have loops, and can just write code -- so inside the macro we just expand the output to a tuple which is large enough to hold all of the outputs.</p> What does join!</code> return?</h2> The definition of async_macros::join!</code> is fairly brief, so I'll just share it right here. The only detail missing is the defition of the MaybeDone</code> type: it's a wrapper which can be awaited, and stores the output type of the future once it completes. We wait for all instances of MaybeDone</code> to complete, and at the end we take all their values and return it from the future:</p> #[</span>macro_export</span>] </span>macro_rules! </span>join { </span> ($(</span>$fut</span>:</span>ident</span>), $(,)?) => { { </span> async { </span> $( </span> </span>// Move future into a local so that it is pinned in one place and </span> </span>// is no longer accessible by the end user. </span> </span>let mut </span>$fut </span>= </span>$crate</span>::MaybeDone::new(</span>$fut</span>); </span> )* </span> </span>$crate</span>::utils::poll_fn(</span>move </span>\|cx\| { </span> </span>use </span>$crate</span>::utils::future::Future; </span> </span>use </span>$crate</span>::utils::task::Poll; </span> </span>use </span>$crate</span>::utils::pin::Pin; </span> </span> </span>let mut</span> all_done = </span>true</span>; </span> $( </span> </span>let</span> fut = </span>unsafe </span>{ Pin::new_unchecked(&</span>mut </span>$fut</span>) }; </span> all_done &= Future::poll(fut, cx).</span>is_ready</span>(); </span> )* </span> </span>if</span> all_done { </span> Poll::Ready(($( </span> </span>unsafe </span>{ Pin::new_unchecked(&</span>mut </span>$fut</span>) }.</span>take</span>().</span>unwrap</span>(), </span> ))) </span> } </span>else </span>{ </span> Poll::Pending </span> } </span> }).await </span> } </span> } } </span>} </span></code></pre> As you can see the outer-most value returned is an async {}</code> block. This isn't a specific type, but can be referred to using impl Future</code>. However the type returned by Future::join</code> is a concrete Join</code> future. This type can</em> be addressed by name, and actually passed around.</p> However as we chain Future::join</code> repeatedly, the resulting future's signature will look somewhat like: Join<Join<Join<Join<T>>>></code>. This is not great.</p> So on the one hand we have anonymous futures which can only be addressed through impl Future</code>. And on the other hand we have deeply nested futures which are a pain to write by hand. Can we do better?</p> consistent return types</h2> One thing I mentioned at the start but didn't dive in yet is the fact that we'd like to align the return types of join!</code> and Future::join</code>. Even if Future::join</code> would only ever take one other future as an argument, being able to switch between the method and the macro without needing to change the signature of the returned types is a a huge bonus.</p> After having worked on async-std</code> for the past two years where a lot of APIs use async fn</code>, I'm now somewhat convinced that the stdlib should never</em> do this. Which you can see reflected in APIs such as std::future::ready</code> which now returns the concrete future std::future::Ready</code>, whereas in async-std</code> it was just an async fn</code>.</p> Probably another point worth touching on is the futures-rs</code> implementation of join!</code>. This doesn't return any kind of future at all, wrapping the .await</code> call within the macro instead. I feel somewhat strongly that .await</code> calls shouldn't be hidden in code, but instead always be visible.</p> // Example of futures_rs::join! </span>let</span> a = future::ready(</span>1</span>u8</span>); </span>let</span> b = future::ready(</span>2</span>u8</span>); </span>assert_eq!(join!(a, b), (</span>1</span>, </span>2</span>)); </span>// is this sync or async? </span></code></pre> This slight digression into futures-rs</code> aside, I think if we were to add future::Future::join</code> and future::join!</code> functions to the stdlib, both should be returning concrete futures. And because they effectively do the same thing, we should make it so they both return the same Join</code> type.</p> Maybe const can help?</h2> So the question now becomes: "how can we do this?". And I think the answer for this is: "const tuples may be able to help".</p> So const tuples don't exist in Rust today yet. Not even on nightly. The only way to create variadic tuples is through macros like we've shown. However from talking to members of the const-eval WG const tuples are definitely on the roadmap, though it may take a while. However now that we're seeing a move to fund more people to work on the compiler, I'm hoping that this may be possible within a few years, which isn't that</em> long in the grand scheme of things.</p> Given there's no proposal for const tuples, it's hard to write an example since I have no clue what the syntax for it will be. For N-length arrays the syntax is the following:</p> pub fn </span>array_windows</span><</span>const</span> N: </span>usize</span>>(&</span>self</span>) -> ArrayWindows<'_, T, N>; </span></code></pre> The const N: usize</code> here is the length argument for the array of type T</code>. The operations this function returns are on [T; N]</code>. However tuples don't have a consistent type T</code>; values contained within tuples are heterogenous. So a tuple of length N</code> can contain N</code> different types. I have no clue how this would be expressed in const</code> contexts (if at all possible?).</p> So for now let's just pretend we can define N-length tuples inside</em> function signatures, and cross our fingers that this makes enough sense that the idea comes across. Assuming something like that would work, I would expect Future::Join</code> to be able to defined along these lines:</p> impl </span>Future { </span> </span>/// Join with one other future. </span> </span>pub fn </span>join</span><F: Future>(</span>self</span>, </span>other</span>: F) -> Join<(</span>Self</span>, F)>; </span>} </span></code></pre> This signature tries to convey: this future holds at least two futures: Self</code>, and another future we're joining with. For the join!</code> macro we could fill out the types using code generation, populating the values of the tuple at compile time. Invoking it would yield the following return type:</p> let</span> a = future::ready(</span>1</span>u8</span>); </span>let</span> b = future::ready(</span>2</span>u8</span>); </span>let</span> fut = join!(a, b); </span>// fut: Join<(Ready<Output = u8>, Ready<Output = u8>)> </span> </span>let</span> a = future::ready(</span>1</span>u8</span>); </span>let</span> b = future::ready(</span>2</span>u8</span>); </span>let</span> c = future::ready(</span>3</span>u8</span>); </span>let</span> fut = join!(a, b, c); </span>// fut: Join<(Ready<Output = u8>, Ready<Output = u8>, Ready<Output = u8>)> </span></code></pre> Assuming once we have const tuples we'll also have const panic, we can guard against the case where zero futures are provided to Join</code>. Or perhaps the signature should instead be:</p> Join<</span>Self</span>, (Other, Other2)> </span></code></pre> The details are unclear, because well, I don't know how this should work in the future. Maybe there's a different feature a play here too: what if expressing this in signatures actually requires const-variadics</code> or something. This may actually be relying on a variety of features I'm not tracking.</p> What does that mean for adding futures concurrency to the stdlib?</h2> This post is rooted in research I was doing exactly to answer that question. join!</code> and Future::join</code> feel like they do exactly what they should, module some issues around their return types. Unfortunately however it seems the best solution would require a const-eval feature that doesn't even have an RFC yet.</p> Given I expect Rust to stick around for at least a few more decades, and how core this functionality is for async programming. I think it's actually worth waiting to implement these features correctly, rather than rushing to add them in the short term. Libraries such as async-std</code> and async-macros</code> can provide suitable solutions through user space in the interim.</p> In addition to that, there's one more feature in the language required before we can consider adding Future::join</code>: namely, we need either #[cfg(accessible)]</code></a>, or #[cfg(version)]</code></a>. This is currently a blocker to adding any</em> method on the Future</code> trait. Since the majority of the async ecosystem relies on Ext</code> traits to implement missing functionality, adding a method of the same name to the stdlib would cause ambiguity. So in order to prevent accidental ecosystem breakage, libraries outside of std should gain the ability to detect whether a method has been implemented in the stdlib. Which is what accessible</code> and version</code> are for.</p> However one possibility may be that we add join!</code> based on the async {}</code> block implementation in the near term without adding Future::join</code> as well. That would at least allow us to expose that funcionality from the stdlib, even if in some instances it may not be the most ergonomic.</p> Then later on, once we gain the ability to reason about tuples/variadics in const contexts, we can switch the return type to be Join</code>, and add the Future::join</code> method as well. That way we get a solution in the short term, but still do the right thing in the long term. This depends on async {}</code> being forward compatible with returning a concrete future though. I'm not sure if this has been done before, and the lang team might need to weigh in on that.</p> edit (2020-01-17):</strong> As pointed out here</a> by matthieum, if we had a variadic Join</code> type, there's no reason we couldn't implement join</code> on tuples directly.</p> // Tuple::join </span>let</span> a = future::ready(</span>1</span>u8</span>); </span>let</span> b = future::ready(</span>2</span>u8</span>); </span>let</span> c = future::ready(</span>3</span>u8</span>); </span>assert_eq!((a, b, c).await, (</span>1</span>, </span>2</span>, </span>3</span>)); </span> </span>// Array::join </span>let</span> a = future::ready(</span>1</span>u8</span>); </span>let</span> b = future::ready(</span>2</span>u8</span>); </span>let</span> c = future::ready(</span>3</span>u8</span>); </span>assert_eq!([a, b, c].await, [</span>1</span>, </span>2</span>, </span>3</span>]); </span></code></pre> This would soft-deprecate the need to use future::join!</code> and could probably also be extended to arrays and slices of futures too 1</a></sup>. The question is whether the same Join</code> type could be used for all implementations, since it wouldn't return a tuple but an array or vec instead. This can probably only be answered once designs for the corresponding language features start.</p> ^1</sup>join!</code> on an array is effectively an instance of join_all!</code></a>. In my designs I've mostly relegated join_all!</code> as not being a primitive, instead favoring designs such as TaskGroup</code> and ParallelStream</code> for a collection of N futures since these more often than not will want to be run on a executor anyway. However wanting to join N futures is still nice to be able to do, and implementing Array::join</code> may very well provide a way for us to do so.</p> </div> Other considerations</h2> Everything we've expressed here not only applies to future::Future::Join</code> and future::join!</code>. It applies to the try_join</code>, race</code>, and try_race</code> variants as well. The async_std::future</code></a> docs explain how these types cover the full range of awaiting futures.</p> Additionally the future::join</code> variants only really work well when you know ahead of time how many futures you're going to be awaiting. If the number is dynamic, other constructs should be used. In a future post I may talk about TaskGroup</code>, an adaptation of crossbeam::scope</code> I'm working on, inspired by Swift's upcoming task</code> proposal. But other constructs like parallel-stream</code> and FuturesUnordered</code> already exist as well.</p> Conclusion</h2> In this post we've looked at what it would take to add Future::join</code> and join!</code> to the stdlib where both functions would return the same, named future. One plausible way to achieve this would be through const tuples (and possible const variadics, which may or may not be the same thing).</p> However it may be possible to add future::join!</code> in the near term, and once Rust gains the appropriate language features add Future::join</code> and upgrade join!</code> to use the same Join</code> future. This would enable adding the functionality in the near term, but still achieving the ideal design later on.</p> This post is a bit of an experiment: single draft, Saturday morning writing. I've been doing a lot of research into stabilizing async Rust paradigms recently, and figured I'd share some of the findings along the way. In part for my own reference. But also to communicate needs async Rust may have to members of other Rust teams.</p> Anyway, hope you enjoyed this, and hope you have a good weekend!</p> If you or your company like the work I'm doing, you're welcome to support me through GitHub sponsors</a>. Special thanks to my sponsors: hibbian</a>, milesgranger</a>, romatthe</a>, No9</a>, and several others who prefer to remain anonymous.</em></p> Rust 2021 Thu, 24 Sep 2020 00:00:00 +0000 I am tired. Today was rough. This year has been rough. Everyone around me is tired too. 2020 has been a hot mess, coming right after a not so hot 2019, and a messy 2018. It's not just you. It's everything. It's okay.</p> My wish for Rust 2021 is for you to go easy on yourself. No really. You. If you've done any work on Rust at all, that means you doubly so. Yes really</em>.</p> The core team has tasked themselves with starting a foundation</a>. Laudible, but can we take a sec to realize we're still mid-pandemic. Not that long ago our biggest sponsor cut back drastically. People lost their jobs. Co-workers. Friends. People we love and cherish. Things are tough. And there's still much</a>, much</a>, much more. And right while all this is happening we're creating this foundation. Ooph.</p> That's why I want the core team to take a break. Go on a vacation. Close the office for a while. I don't know when. Maybe do one in the winter. Maybe one in summer, after we have a foundation. Big ol' summer vacation. Too hot to think anyway.</p> Actually not just the core team: I want everyone to take a vacation. The whole Rust project. Hit the brakes, take it easy. Archive rust-lang/rust</code> for a week. Can't come to work if the office is locked. Take a breath. Take a bath.</p> "Rust" is the outcome of the processes we apply. It's through dialog, labor, and mutual respect that we get any of this done. And we can't do these things if we can't talk about how we feel. And uhh, reading the room here, but people don't seem to be doing well. How can we feel safe in dialog if we don't even feel physically safe. We really gotta take care of ourselves first before we take care of RFC2000.</p> Of course we also should find ways to get paid</a>. Of course stdsimd</a> is cool, and error handling</a> is cool, and async</a> is cool, and…</p> Yea.</p> The backlog of things to work on is infinite. The people that work on them aren't. The cool problems to work on will be here when we come back. I just hope you'll still be here with me ♥</p> Async Iteration I: Iteration Semantics Fri, 18 Sep 2020 00:00:00 +0000 This post is part of the Async Iteration series:</em></p> Async Iteration I: Semantics</a> (this post)</li> Async Iteration II: Async Iterator Crate</a></li> Async Iteration III: Async Iterator Trait</a></li> </ol> In a recent lang team meeting</a> we discussed the upcoming Stream</code></a> RFC. We covered both the steps required to land it, and the steps we'd want to take after that. One of the things we'd want eventually for streams is: "async iteration syntax". Just like for x in y</code> works for Iterator</code>, we'd want something similar to work for Stream</code>. For example JavaScript support both sync</a> and async</a> iteration through for..of</code> loops:</p> // sync </span>for </span>(</span>let </span>num </span>of </span>iter</span>) { </span> </span>console</span>.</span>log</span>(</span>i</span>); </span>} </span> </span>// async </span>for await </span>(</span>let </span>event </span>of </span>emitter</span>) { </span> </span>console</span>.</span>log</span>(event); </span>} </span></code></pre> However when talking about async iteration syntax in Rust one question has come up repeatedly:</p> Should async iteration be parallel or sequential by default?</p> </blockquote> In this post I'd like to cover the challenges I see with parallel async iteration, and make the case for, spoilers, defaulting to sequential iteration semantics.</p> However before we continue, the usual disclaimers: I'm not on the lang team. I do not hold any direct decision making power about the Rust language. I am a member of the Async Foundations WG, but I do not speak for anyone on the team other than myself.</strong> This post is from my perspective only.</p> The problem space</h2> So the core question we're trying to answer is: "should async iteration default to sequential or parallel?" Putting this into code, it's a question of what some hypothetical future "async iteration" syntax should compile down to, and which semantics are provided:</p> // This code... </span>for</span> event.await in channel { </span> dbg!(event); </span>} </span> </span>// ... should it lower into this? (sequential) </span>while let </span>Some(event) = channel.</span>next</span>().await { </span> dbg!(event); </span>} </span> </span>// ... or should it lower into this? (parallel) </span>channel.</span>into_par_stream</span>().</span>for_each</span>(async \|event\| dbg!(event)); </span></code></pre> The "lowering" we're describing here is not the precise</em> code we'd expect the compiler to output. But hopefully it gets the point across. In the "sequential" case we'd expect async iteration loops to behave much the same as async while let Some()</code> loops do today. And in the parallel case, we'd expect something to work with semantics similar to those of parallel-stream</a>.</p> The main argument I've heard in favor of parallel async iteration semantics is that in most cases it's what people want anyway. For example take the following code:</p> use </span>async_std::io; </span>use </span>async_std::net::TcpListener; </span>use </span>async_std::prelude::; </span> </span>let</span> listener = TcpListener::bind("</span>127.0.0.1:8080</span>").await?; </span> </span>for</span> stream.await? in listener.</span>incoming</span>() { </span> io::copy(&stream, &stream).await?; </span>} </span></code></pre> Under sequential iteration semantics this loop would process at most 1 TCP connection at the time, irregardless of system resources available. But under parallel iteration semantics, this will have access to all resources available. The idea is that parallel iteration semantics provide a language level solution to this problem.</p> Control flow operators</h2> The first issue that arises with parallel async iteration is that the break</code> and continue</code> control flow operators will have different semantics, or need to be disallowed altogether. Take the following code:</p> use </span>async_std::io; </span>use </span>async_std::net::TcpListener; </span>use </span>async_std::prelude::*; </span> </span>let</span> listener = TcpListener::bind("</span>127.0.0.1:8080</span>").await?; </span> </span>// Let's pretend this is valid syntax.. </span>for </span>(i, stream?).await in listener.</span>incoming</span>().</span>enumerate</span>() { </span> </span>if</span> i === </span>1000 </span>{ </span> </span>break</span>; </span> } </span> io::copy(&stream, &stream).await?; </span>} </span></code></pre> Say we have a few dozen parallel requests open when we hit break</code>. We should probably stop accepting new requests. What should happen to the still open connections?</p> There is no right answer. We may want to drop them. We may want to keep them running. More likely the desired behavior may depend on what you're doing. But the semantics will always</em> be different than in regular for loops because</em> of parallelism.</p> Adding new semantics to a well-established concept is tricky, and may actually lead to subtle translation issues when converting existing algorithms to async Rust. The safest option will be to disallow these keywords in this context altogether.</p> Consistency with non-async Rust</h2> We may want to think of "parallel iteration semantics" as something unique to async Rust, but in fact non-async Rust faces very similar challenges. To give a very literal example, the TCP example we used earlier would not be parallelized in non-async Rust as well:</p> use </span>std::io; </span>use </span>std::net::TcpListener; </span> </span>let</span> listener = TcpListener::bind("</span>127.0.0.1:8080</span>").await?; </span> </span>for</span> stream in listener.</span>incoming</span>() { </span> </span>let</span> stream = stream?; </span> </span>let </span>(reader, writer) = &</span>mut </span>(&stream, &stream); </span> io::copy(reader, writer)?; </span>} </span></code></pre> Networking in non-async Rust is not the best match, but for example a data processing pipeline would be. And if we forget to parallelize at the right points there, we'll face very similar issues.</p> The point being: I don't think "parallelism" is a problem unique to async Rust. And because of that I don't think we should look for a solution that's specific to async Rust, but instead look for a solution that can be applied across domains. I think RFC 2930 (buf_read</code>)</a> is a great example of an issue that was found in async Rust, but was determined to apply to non-async Rust as well, and for which a solution was found that can be applied to both domains.</p> How would tasks be spawned?</h2> In order for tasks to be parallelized, they would need to be spawned on a thread pool somehow. Which means a crucial step in enabling parallel iteration at the language level is to have a notion of an executor as part of the language.</p> C++ is in the process of adding</a> to the language (estimate is C++23), so it's not unthinkable for Rust to do the same. In fact Boats has written about adding a #[global_executor]</code> before</a>.</p> // The lowered code from the previous example... </span>channel.</span>into_par_stream</span>().</span>for_each</span>(async \|event\| dbg!(event)); </span> </span>// ... would in turn lower into this </span>while let </span>Some(event) = channel.</span>next</span>() { </span> task::spawn(async </span>move </span>{ </span> dbg!(event); </span> }); </span>} </span></code></pre> As you can see we're calling task::spawn</code> for each event. That functionality would need to come from somewhere, and because async iteration is a language level concern this would mean that functionality would need to become part of the language.</p> I think having a way to perform parallel iteration at the language level is a great</em> idea. And having a #[global_executor]</code> hook with a default implementation would solve a lot of issues the ecosystem currently has.</p> But if history is any indicator this would all take quite a while to shake out. And I think making async iteration syntax depend on this landing (even if unstable) would mean it might take a lot longer.</p> Parallelizing !Send streams</h2> Not all streams are Send</code>, and async iteration should be able to work with this as well. We can't quite</em> propagate !Send</code> the way we do for async blocks; the global executor would need to support spawning !Send</code> futures as well. This is not a major hurdle as for example async-stc provides spawn_local</code></a> function. But it's part of the design space to consider.</p> Backpressure & concurency control</h2> For all its warts something that Node.js streams got right was backpressure</em>. If for example writing to a file is slower than reading from a file, the pipeline won't keep reading data if the writer isn't available. This ensures your RAM doesn't fill up with bytes that are waiting to be written.</p> const </span>fs </span>= </span>require</span>('</span>fs</span>'); </span> </span>const </span>reader </span>= </span>fs</span>.</span>createReadStream</span>('</span>file.txt</span>'); </span>const </span>compress </span>= </span>zlib</span>.</span>createGzip</span>(); </span>const </span>writer </span>= </span>fs</span>.</span>createWriteStream</span>('</span>file.txt.gz</span>'); </span> </span>reader</span>.</span>pipe</span>(</span>compress</span>).</span>pipe</span>(</span>writer</span>); </span></code></pre> In Rust sequential async iteration applies backpressure by default. A translation of the above Node.js code would be somewhat like this:</p> let</span> reader = fs::open("</span>file.txt</span>").await?; </span>let</span> writer = fs::open("</span>file.txt.gz</span>").await?; </span> </span>for</span> buf in GzipEncoder::new(reader) { </span> writer.</span>write</span>(buf).await?; </span>} </span></code></pre> async-compression</code>'s Stream</code> types</a> already work somewhat like this. Because we're performing sequential async iteration here, reading data from the stream is constrained by the speed at which we can write. But also vice-versa: the stream is paused if no new data is available to be read.</p> If async iteration syntax was parallel by default we would lose backpressure by default. Which means if no constraints are specified, by default parallel async iteration will keep reading data even if there is no consumer ready to process the data</strong>. Instead limits will always need to be manually set:</p> for</span> buf in GzipEncoder::new(reader).</span>limit</span>(</span>5</span>) { </span> writer.</span>write</span>(buf).await?; </span>} </span></code></pre> There is a third solution to backpressure we haven't discussed yet: set some</em> limit by default. Instead of async iteration having unbounded parallelism by default, we could say, enable 1000</code> items by default. This limit will however almost always be wrong, and really should be set with knowledge of the workload.</p> name</th> max throughput</th> backpressure?</th></tr></thead> sequential</td> 1</td> yes</td></tr> bounded</td> n</td> yes</td></tr> unbounded</td> unlimited</td> no</td></tr> </tbody></table> How common is parallel iteration?</h2> A large part of the argument in favor of parallel semantics by default for async iteration is that it is the most commonly desired semantics. Perhaps that may be true today within certain domains; but how true do we expect that to be in the future for across all domains async Rust will cover?</p> If we want to get a sense of where async Rust might be headed we can take a look at Node.js streams: they've been around for a decade longer than Rust streams have. A common kind of stream in the Node.js ecosystem is the transform stream</a>: a stream that takes an input stream and produces an output stream. Take for example the csv-parser package</a>: it takes a stream of CSV text, and produces an output stream of parsed elements:</p> NAME</span>,</span>AGE </span>Daffy Duck</span>,</span>24 </span>Bugs Bunny</span>,</span>22 </span></code></pre> const </span>csv </span>= </span>require</span>('</span>csv-parser</span>') </span>const </span>fs </span>= </span>require</span>('</span>fs</span>') </span>const </span>results </span>= []; </span> </span>fs</span>.</span>createReadStream</span>('</span>data.csv</span>') </span> .</span>pipe</span>(</span>csv</span>()) </span> .</span>on</span>('</span>data</span>', (</span>data</span>) </span>=> </span>results</span>.</span>push</span>(</span>data</span>)) </span> .</span>on</span>('</span>end</span>', () </span>=> </span>{ </span> </span>console</span>.</span>log</span>(</span>results</span>); </span> </span>// [ </span> </span>// { NAME: 'Daffy Duck', AGE: '24' }, </span> </span>// { NAME: 'Bugs Bunny', AGE: '22' } </span> </span>// ] </span> }); </span></code></pre> Translating this to async Rust (with async iteration syntax) we could have something like this:</p> use </span>async_std::fs::File; </span>use </span>csv_parser::Csv; </span>use </span>std::io; </span> </span>async </span>fn </span>main</span>() -> io::Result<()> { </span> </span>let</span> file = File::open("</span>data.csv</span>").await? </span> </span>for</span> item.await? in Csv::new(file) { </span> println!("</span>name: </span>{}</span>, age: </span>{}</span>", item["</span>NAME</span>"], item["</span>AGE</span>"]); </span> } </span>} </span></code></pre> In this case throughput is likely going to be limited by how fast we can read from disk, which means spawning tasks for the inner logic is likely not going to speed this up. In fact, spawning a task per item may in fact slow down the parser since the logic of "read the file" and "operate on the file" can no longer be compiled down into a single state machine</strong> 1</a></sup>. While tasks may be cheap compared to threads, they're by no means without a cost.</p> ^{1</sup> Inlining performance really depends on the workload. But as a rule of thumb: if something can't be inlined we're likely going to miss out on optimizations. Obviously "println" isn't a good example for something that can be inlined, but you get the gist.</p> </div> The example above could be parallelized, if say, we were processing a directory of files. We could then spawn a task per file. But each file itself would still be processed as a sequential stream.</p> Another example of sequential stream semantics is the HTTP/1.1 protocol: the stream of incoming TCP connections is parallelized, but converting the TCP stream into a sequence of HTTP requests is sequential. And within the request itself, each HTTP body is sequential as well. Even a multiplexing protocol such as HTTP/2 eventually breaks down into request / response pairs which have sequential semantics:</p>}async </span>fn </span>main</span>() -> io::Result<()> { </span> </span>// parallel </span> </span>for</span> stream.await? in TcpListener::accept("</span>localhost:8080</span>") { </span> </span>// parallel </span> </span>for</span> req.await? in Http2Listener::connect(stream) { </span> </span>// sequential </span> </span>for </span>(i, line?).await in req.</span>lines</span>() { </span> println!("</span>line: </span>{}</span>, value: </span>{}</span>", i, line); </span> } </span> } </span> } </span>} </span></code></pre> While parallel iteration may be common, it doesn't seem likely it will ever be used in such an overwhelming amount that sequential iteration is made redundant. It seems most likely people will want to mix both in projects, and we should seek to make both parallel and async iteration convenient.</p> Compiler hints</h2> So far much of this post has been about why "parallel semantics for async iteration" are not the right default. But this doesn't mean that the issue that intends to address isn't worth addressing. Defaulting to parallel semantics is only one tool.</p> What if we could use different tools? For example, one issue raised is: "people might forget to parallelize certain streams, missing out on performance". Could we solve this issue through a lint instead? I think we could, for the most part!</p> For example clippy could detect known instances of core APIs that are should probably be parallelized, and warn when they're not. For example handling TCP connections, or processing files in a directory are things you probably want to do in parallel.</p> Or if we wanted to allow anyone to add it to their APIs: Rust could introduce a #[must_par_spawn]</code> hint that could be added to structs, much like #[must_use]</code>. It could come with a counterpart tag that marks a function as "this spawns tasks". And the compiler could warn if something wasn't spawned 2</a></sup>.</p> ^2</sup>There is an RFC being written for the must_not_await</code></a> hint; going in this direction would not be without precedent.</p> </div> #[</span>must_par_spawn</span>] </span>struct </span>HttpStream; </span> </span>#[</span>par_spawn</span>] </span>async </span>fn </span>spawn</span><T, F: Future<Output = T> + </span>'static</span>>(</span>f</span>: F) -> T; </span></code></pre> Both solutions would need work before implementing; but they present alternate directions that can be explored instead of resorting to parallel async iteration syntax.</p> tool-assisted optimization</h2> In the post Futures and Segmented Stacks</a> Boats talks about techniques to make futures more efficient by moving their stack onto the heap. Locating where to segment your stack is hard to do by hand, but could likely become trivial with the right tooling.</p> Parallelism has many similarities: finding the right places to parallelize can be tricky. Having a tool that can locate where tasks should be spawned, and which tasks aren't needed would be great. Runtime profiling for async programs seems like a generally underexplored domain, and I suspect more and better tooling could get us really far.</p> Parallel iteration syntax</h2> So far in this post we've talked about why async iteration shouldn't default to parallel semantics by default</em>. But I do agree that it would be great if we had some</em> syntax for parallel iteration. In the "future directions" section of my parallel-stream</a> post, I laid out what that could look like (using par for</code>):</p> let mut</span> listener = TcpListener::bind("</span>127.0.0.1:8080</span>").await?; </span>par </span>for</span> stream.await? in listener.</span>incoming</span>() { </span> io::copy(&stream, &stream).await?; </span>} </span></code></pre> In par for</code> loops keywords such as break</code> and continue</code> would be invalid, and each task would be spawned on a task. This syntax could be made to work for both sync and non-async Rust, backed by different execution strategies. And finally, being able to talk about parallelism as a first-class language concept would enable much better diagnostics, documentation, and tooling to be implemented.</p> For example, accidentally writing for x.await</code> instead of par for x.await</code> on a struct tagged as #[must_par_spawn]</code> could be met with the following diagnostic:</p> warning: loop is sequential </span> --> src/main.rs:2:9 </span> \| </span>2 \| for stream.await? in listener.incoming() { </span> \| ^ help: use `par for stream.await? in listener.incoming()` to parallelize </span> \| </span> = note: `#[warn(sequential_loops)]` on by default </span></code></pre> I think parallel iteration syntax in some form would be fantastic</em>. I just would like to see it exist in a similar way for both async and non-async Rust.</p> Addendum: lending streams</h2> Scottmcm</a> pointed out on Zulip that parallel iteration semantics also wouldn't work well with lending streams</a> (also known as "streaming" or "attached" streams).</p> In a lending stream the Item</code> that gets returned by Stream</code> may be borrowed from self. It can only be used as long as the self reference remains live. For example a Stream</code> where:</p> type </span>Item<</span>'iter</span>> = &</span>'iter mut</span> State; </span></code></pre> ...would fail to compile, which seems rather limiting.</p> Conclusion</h2> Parallel iteration semantics for Stream</code> are an incredible concept that has with much potential, but it seems unlikely to be the right fit for all cases.</p> By defaulting to sequential iteration semantics for Stream</code> parity with Iterator</code> is preserved. This not only ensures keywords such as break</code> and continue</code> work in a comparable way, it also enables looking at the design of "parallel iteration" as a design that can be shared between async and non-async Rust.</p> We still have a fair bit to go before we can add async iteration syntax to Rust. The Stream</code> RFC draft still needs to be submitted and accepted. There are still questions about migrations and interop with the stream adapters in the ecosystem. And a design that covers everything from IntoStream</code> to syntax 3</a></sup>. But even though all that may still be a while; hopefully this post can serve to anchor discussion on whether iterating over a Stream</code> should default to sequential or parallel 4</a></sup>.</p> ^{3</sup> All async iteration syntax in this post has been made up, and is by no means an indication of what the lang team will decide on. I just needed something up for the purpose of this post.</p> </div>}^{4</sup> I really tried making a joke about "If a stream would wear iteration, would it wear it this way, or that way" but couldn't get it to work. My biggest regret in publishing this post.</p> </div> Thanks to Friedel Ziegelmayer and Florian Gilcher for proof reading.</em></p>} Notebooks Mon, 27 Jul 2020 00:00:00 +0000 Ever since I picked up programming in 2013 I haven't really stopped. Weekdays, weekends, evenings. I've spent much of my adult life obsessing about programs. But that's come at a cost: over the years I've started to realize what I've been trading for the time spent working on projects. And I realized I needed to change course.</p> Desiring change is a useful starting point; but it needs to be followed up by execution. My default mode of doing things was: "drop everything, focus on the new thing", but that was the root of the problem I was trying to address 1</a></sup>. I realized this wasn't enough, and what I needed was the ability to formulate plans and the ability to review.</p> The answer for me has been to adopt a framework: a notebook, pen, and ruler. In this post I want to share more on how that's worked for me.</p> ^1</sup>I made a list of things I wanted to do</a> in Jan 2018. That year I only managed to do the first thing on the list. My 2018 GitHub contribs</a> are a testament to the absurd amount of time spent. Looking back at this list two years later was confrontational and made me realize that I needed to actively change my approach if I ever wanted to make progress on the things I cared about.</p> </div> bullet journal: a paper OS</h2> In January I first learned about "bullet journalling" through the Pick Up Limes channel on YouTube</a>. The whole video is worth a watch, but this passage really stood out to me:</p> I genuinely didn't know how I functioned before this system. Because I was someone who had notes everywhere. [...] It always felt like there was information coming at me from every direction, but at the same time I couldn't really find anything when I was looking for it. But now, with this system, I feel like I've got a little built-in secretary always reminding me of what I need to do.</p> </blockquote> Someone was telling me that a notebook</em> could act as a secratary? That sounded amazing, and I wanted to learn more! 2</a></sup> So I read the book "The Bullet Journal Method"</em>. In it the author Ryder Carroll lays out a framework for taking notes he describes as an "OS for managing your life".</p> The core tennets of the system are:</p> A system to quickly take notes and categorize them ("rapid logging").</li> A time structure to organize lists by (daily, monthly, yearly).</li> Key moments when and how to interact with the journal.</li> </ul> A key reason why this has to be on paper is that the expression of these ideas is inherently personalized. Every person has different requirements for what they need to track, and a physical notebook is about the only medium that can provide this degree of flexibility. It's a system that serves the needs you have here and now, and adapts to your needs over time.</p> Another quality of paper is that every action on it takes effort. You cannot automate a notebook; the most convenient you can make it is to create a framework. This ensures you have an active relationship with your notes. Nothing will automatically be added. Nothing will automatically be carried over. Every entry is deliberate. And this provides opportunity to reflect and prioritize.</p> ^{2</sup> Admittedly I had to overcome a kneejerk reaction to dismiss paper as the right interface to take notes. I suspect this may be due to some ingrained tech exceptionalism, but I went into it thinking: "How can I turn this into an app later?" The answer is: it wouldn't work well. Paper is infinitely versatile and allows adapting layouts to be best suited for you, at a given time. It's an interface that evolves with you. And no app can replace that.</p> </div>}Tools</h2> It's possible to bullet journal as long as you have a notebook and pen. But these are the tools I use:</p> Notebook: LEUCHTTURM1917 A5 hardcover dotted</a> — 18 EUR</li> Pen: Stabilo point 88, fine 0.4mm</a> — 2 EUR</li> Ruler: Rumold transparent</a> — 1 EUR</li> </ul> I estimate I go through about 3 notebooks per year, estimating my equipment cost to be about 6 euros per month. I also highly recommend using a transparent ruler if you can find one, as I found it easier to use than opaque variants.</p> Journaling in practice</h2> Did I say journals are customized to the owner's needs? There is no right</em> way to journal; but here's how I do it.</p> The blank spread</h2> Each page in the journal is empty when you start out. I was so daunted by it the first time I started: I was afraid of making mistakes and doing things wrong. A blank page has no system yet; no relationship to any other page either. And that's okay. We get to define that, and the most important part is to start.</p> Daily log</h2> Every Sunday I plan my week ahead. I create a new daily log spread, and I copy over all dates and tasks for the week over from my monthly log. And I copy over the tasks from the week before that I need to do this week. This means when Monday rolls around I don't start at a deficit, but instead already know what I need to work on.</p> My daily journal is a two-page spread that starts on Monday and ends on Sunday. This leaves 1/8th of the pages over to write notes, which I usually use to note extra tasks for the week, or tasks to carry over to the next week.</p> I start each day with a list of things I need to do (including meetings and things to read). And end the day with a list of things I have done. If I've missed something I carry it over to another day, or file it in my monthly log.</p> Monthly log</h2> The monthly log provides a wider breakdown of the month. These are the 14 pages after the yearly log. That is 6 months + 2 pages to capture events after that. The idea is that we'll run out of notebook pages before we reach the end of the 6-month period.</p> On the left hand of each page there's a line per day, with little markings to delimate each week. And on the right side there's space for todos for the month. I like to plan this spread out at the start of the month. I filter and carry over uncompleted tasks from the prior month. And copy over entries from the yearly log.</p> The monthly log is also a good place for habit tracking. On the right side of the left page I have a column marked "e" (for "exercise") that I add a small x on the day of whenever I've done exercise. This allows me to see at a glance how I've been doing (17 entries so far this month).</p> I've recently started experimenting with task prioritization on the right page. On the bottom half of the page I create two columns: one with things (projects?) I'm currently working on. And one with goals (projects?) I would like to be doing. Assuming I need to drop something in order to make time for something new, I can then figure out how to break it down further.</p> Yearly log</h2> The first 4 pages in any notebook make up the yearly log. This is an overview of each month; with 3 months per page. Here I write down important events, anniversaries, birthdays, and major upcoming events. I fill this out whenver I get a new notebook, filtering and carrying over the yearly log from an earlier notebook. It also serves to plan year-scale events such as holidays on.</p> When first starting a notebook this is a fun section to start with. Look up national holidays. Ask friends when their birthdays are. Schedule and spread out vacations. Look up conference days, book releases, movie releases.</p> I usually only consult the yearly log at the start of the month to carry over events to the monthly log, and whenever I find out about a friend's birthday 3</a></sup>. But hopefully I'll find some other uses for the yearly log in the future — this is a useful timescale to look at things through, and one I don't nearly do enough.</p> ^{3</sup> For any pal reading this who's casually mentioned their birthday. I've been keeping track, hehe.</p> </div> Custom: Daily exercise log</h2> The spreads I've covered so far are fairly standard. They're almost 1:1 implementations of Ryder Carroll's base templates. I like them because they're easy to create and take minimal upkeep.</p> But one project that didn't quite fit in that mold was tracking and planning exercise. Or well; the way I solve it was by creating another weekly log that I use to plan and track exercise in.</p> At the front of my notebook I reserve two pages for for exercises. The left page contains a collection of sets I can do. And the right page individual exercises catagorized per muscle group. This much resembles the meals/groceries spread, but for exercise.</p> Custom: daily meals/groceries</h2> I realized last year that my partner and I were ordering in food several times a week. The reason for that was because we couldn't find the time to cook, even for fun on the weekends. So I wanted to find a way to lower the barrier.</p> The answer ended up being a combination of several things: decide what to cook ahead of time, make sure we're stocked, and keep a clean cooking surface at all times. This is still a work in progress, but being able to plan meals and groceries is a useful tool to have.</p> The way we do this is by creating a spread with on the left-hand side an overview of meals for the week: per-day breakdown at the top, and individual recipes at the bottom. And an overview of ingredients on the right-hand side.</p> On a good week I sit down with my partner and talk about what we want to cook that week. Then I do a full index of what we need to prepare those meals with. I then cross out what we already have in the house from the list. And finally take a look at the list of staples in the front of the notebook and check whether we're missing any. We then end up with a decent list of things we need to buy for the week, and are ready to do a grocery run.</p> Sometimes we still unexpectedly make plans with pals, or have meetings run over that prevent cooking meals. But if planned well you roll over meals, or postpone them for the week after. This takes some practice though.</p> Figuring out what to buy from the store can be a tedious process. So I like to keep a list of staples in the front of the notebook (somewhere after the monthly logs, before the first daily log). It's a useful reference that allows me to quickly check whether I'm missing something, or I can check it for inspiration on what to cook.</p>}Custom: sleep log</h2> One thing I started tracking earlier this year was my sleeping patterns. I noticed I had trouble falling asleep, trouble waking up, and felt pretty sad a lot of the time. I figured this would be helpful to track. I no longer track it (yay good sleep!), but the image above is my sleep for the month February. 4</a></sup></p> ^{4</sup> Probably the worst day that month was when I hit 4 and then 5 hours of sleep, then went for dinner with friends, and had to organize 20 people to get to a bar. I also caught wind the contract I was negotiating unexpectedly fell through. The whole week was pretty rough. But it probably would've been worse if I hadn't been aware of my sleeping patterns.</p> </div>}Social</h2> Something new I've started, and am still working on, is keeping track of interactions with friends and acquiantances. There are people who I used to interact with regularly but for whatever reason don't interact with enough. And other people who I wish I was closer with, but am not.</p> This requires active planning, tracking, and considering. I know how nice it is when a friend I haven't heard of in a while reaches out — it doesn't have to be much really. I want to be that person for others. The friend who gets in touch; who reaches out.</p> I'm still experimenting with the right system for this, and reading up on material how to be better (I just started "we should get together"</a>). Perhaps I'll figure something out here and have something to share in the future.</p> Tidying</h2> Something I've desired to do for a few years has been to be better around the house. Clean more often, do more chores. Part of this has been communication with my partner. Part doing research on how to actually keep a house tidy (konmari actually worked for me). And much of it I've been able to track through my notebook.</p> I now clean the windows once every 3 months. I've created a daily habit of cleaning up in the mornings. I keep a list of chores to do when I have time available. This has gradually helped me get better around the house, which is not only nice for the people I live with. It makes me feel more calm and in touch with where I spend most of my time.</p> Events</h2> In the two months before the pandemic my partner and I started going to shows more. Every month I'd look up events around Berlin, and we'd plan them in to go see throughout the month.</p> Because it's so easy to go through the same loop every day. Days turn into weeks. Weeks into months. And in my case: it feels like little has changed for me in the last 3 years. Making an active effort to do new things is something I wanted to do.</p> Unfortunately the pandemic has thrown a bit of a wrench into this, but I still wanted to share this actually worked really well for as long as I did it. With lockdown measures decreasing in Berlin I probably should cautiously try something like this again: even if it's just parks to visit, areas to go, or recipes to try.</p> Conclusion</h2> This has been an overview of how I use my notebook to track much of my life. I started doing this 8 months ago, and I feel it's helped me gain more control of my life. Perhaps you'll find this useful too.</p> Further Resources</h2> Bullet Journal on YouTube</a></li> The Bullet Journal Method book</a> (The audiobook is ~4 hours)</li> </ul>}}

Placing Arguments

Wed, 13 Aug 2025 00:00:00 +0000

In my last post I introduced placing functions</a>, which are functions that can “return” values without copying them. This is useful not just because it’s more efficient (fewer copies), but also because it guarantees addresses remain stable - which is something we need for types that have internal borrows (self-referential types). Here is a little reminder of what placing functions look like:</p>

#[</span>placing</span>]                    </span>// ← 1. Marks a function as "placing".
</span>fn </span>new_cat</span>(</span>age</span>: </span>u8</span>) -> Cat {  </span>// ← 2. Has a logical return type of `Cat`.
</span>    Cat { age }               </span>// ← 3. Constructs `Cat` in the caller's frame.
</span>}
</span></code></pre>
As you can see this is just regular Rust code, with the only difference being
the added #[placing]</code> (placeholder) notation. The main idea behind placing
functions is backwards-compatibility</em>, meaning: we should be able to
progressively add #[placing]</code> notations to the entire stdlib, just like we’re
doing with const. Which is important, because we don’t want to add yet another
axis to the language, locking out address-sensitive types from being used with
existing traits and functions. We’re already suffering through the bifurcation
Pin</code> has introduced</a> 1</a></sup>, and I don’t
think we should go through this again for self-referential types.</p>
^{1</sup>
And no, unfortunately the “pin ergonomics” experiment won’t do anything to resolve this problem. I think it’s a shame we’re not taking this problem more seriously, because this kind of incompatibility affects the language in ways that extend far beyond syntax and conveniences.</p>
</div>}Placing Functions</h2>
It’s cool to have ideals and visions for how thing should be, but we do always
need to square that with reality. And while placing functions</em> seem like they
will work without a hitch (fallible placing functions post tbd), it’s placing
arguments</em> which seem like they’re going to be a problem. To recap what placing
arguments are about, the idea is that we would be able to write the following
definition, and things would just work</em> without any copies:</p>
impl</span><T> Box<T> {
</span>    </span>// `Box::new` here takes type `T` and
</span>    </span>// constructs it in-place on the heap
</span>    </span>fn </span>new</span>(</span>x</span>: #[placing] T) -> </span>Self </span>{ ... }
</span>}
</span></code></pre>
In the post I made the following assertion (in bold even, because it’s important):</p>

As a core design constraint: invoking a function that takes placing arguments should be no different from a regular function. This is needed to keep APIs backwards-compatible.</p>
</blockquote>
Cool idea, just one small hitch: we can’t actually do that. And that is because
placing arguments</em> fundamentally encode callback semantics</em>. Olivier Faure
provided this example to prove why:</p>
let</span> x = Box::new({
</span>    </span>return </span>0</span>;
</span>    </span>12
</span>});
</span></code></pre>
To preserve compatibility, we need to preserve order of execution. In Rust today
if you call this, the expression in Box::new</code> will be evaluated before
Box::new</code> is called. In this case that means that this will return before
Box::new</code> has a chance to allocate.</p>
If we applied placing semantics, it would fundamentally need to change the
ordering. Before we can place, the place has to be created. In this case that
place is on the heap, which means interacting with the allocator, which in Rust
can panic. That means that code which looks like it should return</code> before it
calls the function, would actually panic. And that’s bad!</p>
To fully push this point home, Taylor Cramer recently provided an example which
showed how the execution ordering also causes issues with the borrow checker. In this example we would need to hold &vec</code> as part of the expression for vec.len</code>, while also holding &mut vec</code> for vec.push</code>. And that can’t work; not even with view types and abstract fields</a> 2</a></sup>:</p>
^{2</sup>
Both Vec::len</code> and Vec::push</code> need to access the vector’s “length”. Virtualizing fields / partial borrows won’t change that.</p>
</div>}   vec.</span>push</span>(</span>make_big_thing</span>(vec.</span>len</span>()));
</span>// ^^^^^^^^                ^^^^^^^
</span>// | &mut vec              | &vec
</span></code></pre>
Closures</h2>
The only way I see us making sense of this is by actually requiring that users
use closures here. That would make the ordering far clearer, and preventing any
surprises about ordering or borrows. Ralf Jung made the observation that
FnOnce</code> for these kinds of scenarios actually perfectly encodes the semantics
we want, and I agree. Consider this example:</p>
vec.</span>push</span>(|| </span>make_big_thing</span>(vec.</span>len</span>()));
</span></code></pre>
This would give the following error:</p>
error[E0502]: cannot borrow `vec` as mutable because it is also borrowed as immutable
</span>  --> src/main.rs:50:9
</span>   |
</span>47 |   vec.push(|| make_big_thing(vec.len()));
</span>   |                              --- immutable borrow occurs here
</span>...
</span>47 |   vec.push(|| make_big_thing(vec.len()));
</span>   |   ^^^ mutable borrow occurs here
</span></code></pre>
There is nothing specific to placing functions about this error, just like
there’s nothing special about solving it:</p>
let</span> len = vec.</span>len</span>();
</span>vec.</span>push</span>(|| </span>make_big_thing</span>(len));
</span></code></pre>
But we can’t just change the signature of Vec::push</code> to take closures. Instead
we would need to introduce a new method, e.g. Vec::push_with</code> that can take a
closure and place its outputs:</p>
impl</span><T> Vec<T> {
</span>    </span>pub fn </span>push_with</span><F>(&</span>mut </span>self</span>, </span>f</span>: F)
</span>    </span>where
</span>        F: #[placing] FnOnce() -> T;
</span>}
</span></code></pre>
And this points us to a rather important challenge with these APIs, because for
all intents and purposes this sets us down the path of deprecating Vec::push</code>.
The Vec::push_with</code> method is more efficient than Vec::push</code>, and beyond some
compat reasons there will be no reason to keep using Vec::push</code>. So people will
naturally begin treating Vec::push</code> as legacy, even if we don’t outright mark
it as such.</p>
Edition-dependent path resolution</h2>
I’m a firm believer that the obvious choice for something should be the right
choice ^{3</a></sup>. It feels bad to scold people for using Box::new</code>, telling the that they
should be using Box::new_with</code> instead. Or instead of calling
Hashmap::insert</code>, they should be calling Hashmap::insert_with</code>. And so on.</p>}
^3</sup>While not quite the same, you can see this same idea of “the default way should be the right way” reflected in the safety practice of poka-yoke</a>. Or in Hollnagel’s distinction between Safety 1 and Safety 2</a>.</p>
</div>
My preferred way of resolving this would be to have a notion of
edition-dependent paths</em>: imports and symbols that resolve differently
depending on which edition you’re compiling on. For example on edition 2024</code>
and below, people would see both Vec::push</code> and Vec::push_with</code>:</p>
impl</span><T> Vec<T> {
</span>    </span>pub fn </span>push</span>(&</span>mut </span>self</span>, </span>item</span>: T);
</span>    </span>pub fn </span>push_with</span>(&</span>mut </span>self</span>, </span>f</span>: impl #[placing] FnOnce() -> T);
</span>}
</span></code></pre>
But on edition 2024 + 1</code> we would be able to deprecate Vec::push</code> and rename it to
something else (e.g. push_without</code>), and have Vec::push_with</code> take its place:</p>
impl</span><T> Vec<T> {
</span>    </span>// Is called `push_with` on editions 2024 and below
</span>    </span>pub fn </span>push</span>(&</span>mut </span>self</span>, </span>f</span>: impl #[placing] FnOnce() -> T);
</span>    
</span>    </span>// Is called `push` on editions 2024 and below
</span>    #[</span>deprecated</span>(note = "</span>please use `push` instead</span>")]
</span>    </span>pub fn </span>push_without</span>(&</span>mut </span>self</span>, </span>item</span>: T);
</span>}
</span></code></pre>
Under the hood these functions would still resolve to the same symbols after
accounting for editions. Conceptually what this needs is a stable,
edition-independent identifier that libraries can map back onto in
edition-specific ways:</p>
stable identifier</th> edition 2024</th> edition 2024 + 1</th></tr></thead>

::std::vec::Vec::push</code></td> Vec::push</code></td> Vec::push_without</code> †</td></tr>
::std::vec::Vec::push_with</code></td> Vec::push_with</code> †</td> Vec::push</code></td></tr>
</tbody></table>
†: These names are for illustrative purposes only; they are not concrete proposals.</em></p>
And we could even go further with this, where in a hypothetical edition 2024 + 2</code> we might remove the old API entirely, so you would only ever be able to use the new API:</p>
impl</span><T> Vec<T> {
</span>    </span>// Is called `push_with` on editions 2024 and below;
</span>    </span>// `push_without` is no longer available.
</span>    </span>pub fn </span>push</span>(&</span>mut </span>self</span>, </span>f</span>: impl #[placing] FnOnce() -> T);
</span>}
</span></code></pre>
The main reason why we don’t have a system like this already is because it would
be a lot of work and there are guaranteed to be edge cases that make this harder
than you’d assume it is. The basic design I’m describing here is something
people have thought of before, and that’s not the reason it hasn’t happened yet.</p>
However useful edition-dependent path resolution would be in the long run, we
don’t need it out of the gate. We can begin by adding placing variants of
existing methods to the stdlib. It’s probably enough to proceed with, knowing edition-dependent path resolution might be possible in the future.</p>
Thanks to Alice Ryhl for reviewing this post.</em></p>

placing functions
Tue, 08 Jul 2025 00:00:00 +0000
What are placing functions?</h2>
About a year ago I observed that in-place construction seems surprisingly
simple</a>.
By separating the creating of the place in memory from writing the value to
memory, it’s not that hard to see how we can turn that into a language feature.
So about six months ago, that’s what I went ahead and did and created the placing crate</a>: a proc-macro-based prototype for “placing functions”.</p>
Placing functions are functions whose return type is constructed in the caller’s
stack frame rather than in the function’s stack frame. This means that the
address from the moment on construction is stable</em>. Which may not only lead to
improved performance, it also serves as the foundation for a number of
useful features like supporting dyn</code> AFITs (Async Functions in Traits) 1</a></sup>.</p>
In this post I’ll be explaining how placing functions desugar, why placing
functions are the right solution for emplacement, and how placing functions
integrate into the language. Not in as much detail as I’d do for an actual RFC,
but more as a broad introduction to the idea</em> of placing functions. And to get
right into it, here is a basic example of how this would work:</p>
struct </span>Cat {
</span>    </span>age</span>: </span>u8</span>,
</span>}
</span>
</span>impl </span>Cat {
</span>    #[</span>placing</span>]       </span>// ← Marks a function as "placing".
</span>    </span>fn </span>new</span>(</span>age</span>: </span>u8</span>) -> </span>Self </span>{
</span>        </span>Self </span>{ age } </span>// ← Constructs `Self` in the caller's frame.
</span>    }
</span>
</span>    </span>fn </span>age</span>(&</span>self</span>) -> &</span>u8 </span>{
</span>        &</span>self</span>.age
</span>    }
</span>}
</span>
</span>fn </span>main</span>() {
</span>    </span>let</span> cat = Cat::new(</span>12</span>); </span>// ← `Cat` is constructed in-place.
</span>    assert_eq!(cat.</span>age</span>(), &</span>12</span>);
</span>}
</span></code></pre>
A basic desugaring</h2>
The purpose of the placing</code> crate is to prove that placing functions should not
be that hard to implement. I managed to implement a working prototype in a few hours over my winter holidays. In total I've spent maybe four or so days on the implementation. I’m not a compiler engineer though, and I expect that the lovely folks on
T-Compiler can probably recreate this in a fraction of the time it took me.</p>
Since I don't know my way around the rustc frontend, I implemented the placing crate entirely using proc macros. The upside was that I could get something working more quickly. The downside is that proc macros don’t have access to type information, so I had to hack around that limitation. Which results in an API that requires a lot of proc macro attributes. But that’s ok for a proof of concept.</p>
Let’s walk through the basic example I
showed earlier, but this time using the proc macros. Starting by installing the
placing crate:</p>
$</span> cargo add placing
</span></code></pre>
We can then import placing</code> and define our main struct Cat</code>. We need to
annotate this with the #[placing]</code> attribute macro because we need to change
the internal representation slightly. Here is what this looks like:</p>
//! Original
</span>
</span>use </span>placing::placing;
</span>
</span>#[</span>placing</span>]
</span>pub struct </span>Cat {
</span>    </span>age</span>: </span>u8</span>,
</span>}
</span></code></pre>
Let’s take a look at what the #[placing]</code> annotation expands to. As I said in
the intro: placing functions separate the creation of the memory location from
the initialization of the values in said location. For our type Cat</code>, the
location must be of type MaybeUninit<Cat></code>. But because we want to keep the
type the same, even if we change how the internals work, we actually want to
keep Cat</code> as the outer type name and move the fields into an internal
MaybeUninit</code>:</p>
//! Desugared
</span>
</span>use </span>std::mem::MaybeUninit;
</span>
</span>/// This keeps the same external type,
</span>/// but changes the internals to store the fields
</span>/// in a `MaybeUninit`.
</span>#[</span>repr</span>(transparent)]
</span>pub struct </span>Cat(MaybeUninit<InnerCat>);
</span>
</span>/// These are the fields contained in the original
</span>/// type `Cat`. But separated so that they can be
</span>/// wrapped in a `MaybeUninit` internally.
</span>struct </span>InnerCat {
</span>    </span>age</span>: </span>u8</span>,
</span>}
</span></code></pre>
Our desugaring needs to add one last bit to our type definition to ensure it works correctly.
Because we are now holding a MaybeUninit</code> we have to make sure it calls its
destructors when dropped. This means our desugaring  needs to generate a Drop</code> implementation
that calls through the MaybeUninit</code>.</p>
//! Desugared
</span>
</span>impl </span>Drop </span>for </span>Cat {
</span>    </span>fn </span>drop</span>(&</span>mut </span>self</span>) {
</span>        </span>// The constructors guarantee this will never
</span>        </span>// be dropped before it has been initialized.
</span>        </span>unsafe </span>{ </span>self</span>.</span>0.</span>assume_init_drop</span>() }
</span>    }
</span>}
</span></code></pre>
Now that we have our type definition, let’s show how to implement the new</code>
constructor. Because we don’t have access to type information the placing crate
requires annotations on both the impl block and the method:</p>
//! Original
</span>
</span>#[</span>placing</span>]
</span>impl </span>Cat {
</span>    #[</span>placing</span>]
</span>    </span>fn </span>new</span>(</span>age</span>: </span>u8</span>) -> </span>Self </span>{
</span>        </span>Self </span>{ age }
</span>    }
</span>}
</span></code></pre>
This is the trickiest part of the desugaring because it needs to split up the
constructor into two parts. One to create the place, the other to initialize the
values in-place. Conceptually that means rewriting the return type of the last
line into writing into a mutable argument instead.</p>
//! Desugared
</span>
</span>use </span>std::mem::MaybeUninit;
</span>
</span>impl </span>Cat {
</span>    </span>/// Calling this function constructs the place to
</span>    </span>/// initialize the type into. This is part of a
</span>    </span>/// two-part constructor, and it must always be
</span>    </span>/// followed by a call to `new_init`.
</span>    </span>unsafe fn </span>new_uninit</span>() -> </span>Self </span>{
</span>        </span>Self</span>(MaybeUninit::uninit())
</span>    }
</span>
</span>    </span>/// This initializes the values of the type in-place.
</span>    </span>/// `new_init` must not be called more than once, and
</span>    </span>/// must always follow a call to `new_uninit`.
</span>    </span>unsafe fn </span>new_init</span>(&</span>mut </span>self</span>, </span>age</span>: </span>u8</span>) {
</span>        </span>let</span> this = </span>self</span>.</span>0.</span>as_mut_ptr</span>();
</span>        </span>unsafe </span>{ (&raw </span>mut </span>(*this).age).</span>write</span>(age) };
</span>    }
</span>}
</span></code></pre>
Next we also have a getter function. All we need to do with that is teach it how
to reach through the outer struct and into the inner fields. Here is the
definition:</p>
//! Original
</span>
</span>#[</span>placing</span>]
</span>impl </span>Cat {
</span>    </span>fn </span>age</span>(&</span>self</span>) -> </span>u8 </span>{
</span>        &</span>self</span>.age
</span>    }
</span>}
</span></code></pre>
And here is what it expands to:</p>
//! Desugared
</span>
</span>impl </span>Cat {
</span>    </span>fn </span>age</span>(&</span>self</span>) -> </span>u8 </span>{
</span>        </span>let</span> this = </span>unsafe </span>{ </span>self</span>.</span>0.</span>assume_init_ref</span>() };
</span>        this.age
</span>    }
</span>}
</span></code></pre>
With that we’re now ready to create an instance of Cat</code> in-place, and invoke
our getter. This is the only part that can’t be abstracted away, since Rust
macros have very strict scoping rules compared to if we implemented this
directly in the compiler. What we really wish we could do would be this:</p>
let</span> cat = placing!(Cat, new, </span>12</span>);
</span></code></pre>
But for now we have to call this manually instead, and so the invocation looks
like this:</p>
//! Original
</span>
</span>fn </span>main</span>() {
</span>    </span>let mut</span> cat = </span>unsafe </span>{ Cat::new_uninit() };
</span>    </span>unsafe </span>{ cat.</span>new_init</span>(</span>12</span>) };
</span>    assert_eq!(cat.</span>age</span>(), &</span>12</span>);
</span>}
</span></code></pre>
As an aside: in Rust we now have the experimental super_let</code> feature which
makes it possible to construct types in the enclosing scope, but reference them
afterwards. This almost</em> works for our use case, except it can only return
references, not owned types. That means the best we can do with that feature is the following (playground</a>):</p>
#![</span>feature</span>(super_let)]
</span>
</span>macro_rules! </span>new_cat {
</span>    (</span>$value</span>:</span>expr </span>$(,)?) => {
</span>        {
</span>            </span>super let mut</span> cat = </span>unsafe </span>{ Cat::new_uninit()) };
</span>            </span>unsafe </span>{ Cat::new_init(&</span>mut</span> cat, </span>$value</span>) };
</span>            &</span>mut</span> cat </span>// ← ❌ Returns by-ref rather than by-value
</span>        }
</span>    }
</span>}
</span></code></pre>
If we try and return an owned value it actually copies it - which is exactly
what we’re trying to avoid. We might be able to change that in the compiler
implementation. And to summarize: here is the placing</code> crate’s version of our original example:</p>
//! Original
</span>
</span>use </span>placing::placing;
</span>
</span>#[</span>placing</span>]
</span>struct </span>Cat {
</span>    </span>age</span>: </span>u8</span>,
</span>}
</span>
</span>#[</span>placing</span>]
</span>impl </span>Cat {
</span>    #[</span>placing</span>]
</span>    </span>fn </span>new</span>(</span>age</span>: </span>u8</span>) -> </span>Self </span>{
</span>        </span>Self </span>{ age }
</span>    }
</span>
</span>    </span>fn </span>age</span>(&</span>self</span>) -> </span>u8 </span>{
</span>        &</span>self</span>.age
</span>    }
</span>}
</span>
</span>fn </span>main</span>() {
</span>    </span>// `Cat` is constructed in-place.
</span>    </span>let mut</span> cat = </span>unsafe </span>{ Cat::new_uninit() };
</span>    </span>unsafe </span>{ cat.</span>new_init</span>() };
</span> 
</span>    assert_eq!(cat.</span>age</span>(), &</span>12</span>);
</span>}
</span></code></pre>
And here is the same code with all the macros expanded:</p>
//! Desugared
</span>
</span>use </span>std::mem::MaybeUninit;
</span>
</span>#[</span>repr</span>(transparent)]
</span>struct </span>Cat(MaybeUninit<InnerCat>);
</span>struct </span>InnerCat {
</span>    </span>age</span>: </span>u8</span>,
</span>}
</span>
</span>impl </span>Cat {
</span>    </span>/// Creates an uninitialized place
</span>    </span>unsafe fn </span>new_uninit</span>() -> </span>Self </span>{
</span>        </span>Self</span>(MaybeUninit::uninit())
</span>    }
</span>
</span>    </span>/// Initializes the fields in-place
</span>    </span>fn </span>new_init</span>(&</span>mut </span>self</span>, </span>age</span>: </span>u8</span>) {
</span>        </span>let</span> this = </span>self</span>.</span>0.</span>as_mut_ptr</span>();
</span>        </span>unsafe </span>{ (&raw </span>mut </span>(*this).age).</span>write</span>(age) };
</span>    }
</span>
</span>    </span>fn </span>age</span>(&</span>self</span>) -> </span>u8 </span>{
</span>        </span>let</span> this = </span>unsafe </span>{ </span>self</span>.</span>0.</span>assume_init_ref</span>() };
</span>        this.age
</span>    }
</span>}
</span>
</span>impl </span>Drop </span>for </span>Cat {
</span>    </span>fn </span>drop</span>(&</span>mut </span>self</span>) {
</span>        </span>unsafe </span>{ </span>self</span>.</span>0.</span>assume_init_drop</span>() }
</span>    }
</span>}
</span>
</span>fn </span>main</span>() {
</span>    </span>let mut</span> cat = </span>unsafe </span>{ Cat::new_uninit() };
</span>    </span>unsafe </span>{ cat.</span>new_init</span>() };
</span>    assert_eq!(cat.</span>age</span>(), &</span>12</span>);
</span>}
</span></code></pre>
The placing crate also supports desugaring types that return Result</code>, Box</code>,
and Arc</code>. As well as includes support for nesting constructors, where you end
up calling a #[placing] fn</code> from a #[placing] fn</code>. This is why I’m fairly confident this should end up working out.</p>
The main limitation of the crate is that it doesn’t yet support traits. I
started adding support for that, but ended up running out of time. This is the
reason why if you look at the codegen in the crate you’ll see a PLACING: bool</code> const
generic inserted everywhere. It isn’t particularly useful yet, but now you
know why it’s there.</p>
Thinking in placing functions</h2>
Placing functions draw their inspiration from two other language features:</p>

Super Let (Rust Nightly)</strong>: is an experimental feature that enables temporary lifetime extensions</a>. This allows variables to be created in the enclosing scope.</li>
Guaranteed Copy Elision (C++ 17)</strong>: sometimes also called “deferred temporary materialization”</a> guarantees that structs are always constructed in the caller’s scope.</li>
</ul>
If super let</code> operates on block scopes, you can think of placing functions as
operating across function boundaries. And if guaranteed copy elision is an
automated guarantee that applies to all</em> functions, you can think of placing
functions as only ever applying to functions that have been explicitly opted
into this feature.</p>
The design of placing functions attempts to balance three core constraints:</p>

Control</strong>: in certain cases emplacement already happens through optimizations (e.g. inlining). An emplacement language feature must guarantee</em> it emplaces, or else fail compilation.</li>
Integration</strong>: C++’s guaranteed copy elision shows just how broadly applicable emplacement really is. That means the initial upper bound for this language feature is every single function with a return type. That’s incredibly broad, and we need to ensure it integrates closely and easily with the rest of the language.</li>
Compatibility</strong>: Rust’s stdlib makes strong backwards-compatibility guarantees. We will want it to be able to make use of emplacement without</em> breaking backwards compat. This means we cannot mint new traits just to support emplacement, or add new APIs specifically to emplace.</li>
</ul>
It would be a mistake to think of emplacement as a feature with narrow
applicability; C++ provides evidence emplacement is relevant to almost every
constructor</strong>. The right way to think about emplacement is to assume it has a
maximally broad upper bound. But then start designing and implementing a minimal
subset. While we may eventually find limitations or cases where emplacement
isn’t possible; that will be something we prove out through implementation.</p>
This is why I believe we should model “emplacement” more like an effect, and not
like a different kind of language feature. I think of placing functions as
somewhere between const functions and async functions. They change the codegen
of the function, not entirely unlike the generator transform we use for async</code>
and gen</code>. But it never actually lowers to another type we can observe in the
type system, which makes it very similar to const</code>.</p>
Prior Art in Rust</h2>
As I was getting ready to publish this post I ended up talking a little more
with Sy Brand about placing functions, C++, and ABIs. It turns out: Rust already
guarantees emplacement in a number of cases. Consider for example the following code:</p>
pub struct </span>A {
</span>    </span>a</span>: </span>i64</span>,
</span>    </span>b</span>: </span>i64</span>,
</span>    </span>c</span>: </span>i64</span>,
</span>    </span>d</span>: </span>i64</span>,
</span>    </span>e</span>: </span>i64</span>,
</span>}
</span>
</span>impl </span>A {
</span>    </span>pub fn </span>new</span>() -> A {
</span>        A {
</span>            a: </span>42</span>,
</span>            b: </span>69</span>,
</span>            c: </span>4269</span>,
</span>            d: </span>6942</span>,
</span>            e: </span>696942</span>,
</span>        }
</span>    }
</span>}
</span></code></pre>
When we compile this for the SYSV ABI on x86, it outputs the following
assembly (godbolt</a>):</p>
example::A::new::hd00831bc57a4b613:
</span>    </span>mov </span>rax</span>, </span>rdi
</span>    </span>mov </span>qword ptr </span>[</span>rdi</span>], </span>42
</span>    </span>mov </span>qword ptr </span>[</span>rdi </span>+ </span>8</span>], </span>69
</span>    </span>mov </span>qword ptr </span>[</span>rdi </span>+ </span>16</span>], </span>4269
</span>    </span>mov </span>qword ptr </span>[</span>rdi </span>+ </span>24</span>], </span>6942
</span>    </span>mov </span>qword ptr </span>[</span>rdi </span>+ </span>32</span>], </span>696942
</span>    </span>ret
</span></code></pre>
This assembly is writing directly to pointer offsets provided to the function.
In other words: this function is emplacing. And it’s actually guaranteed to do
that, as defined in the x86 SYSV ABI
spec</a>, section 3.2.3:</p>

[!quote]
If the type has class MEMORY, then the caller provides space for the return
value and passes the address of this storage in %rdi</code> as if it were the first
argument to the function. In effect, this address becomes a “hidden” first argument
This storage must not overlap any data visible to the callee through
other names than this argument.
On return %rax</code> will contain the address that has been passed in by the
caller in %rdi</code>.</p>
</blockquote>
A hidden first argument that’s passed to functions? That sure sounds a lot like
how the desugaring for placing functions is intended to work. In fact, C++’s
guaranteed copy elision makes use of this very same feature. Section 3.2.3
states the following:</p>

If a C++ object has either a non-trivial copy constructor or a non-trivial
destructor 11, it is passed by invisible reference (the object is replaced in the
parameter list by a pointer that has class INTEGER) 12.</p>
</blockquote>
This is all incredibly similar to what I’m proposing, but happening
automatically at the ABI level rather than transparently at the language level. It also begs the question:  how easy it would be to modify rustc's ABI lowering code for x64 to just say "if the type was declared with the placing</code> attribute, it's always classified as MEMORY”</em>?</p>
Q&A</h2>
What about placing arguments?</h3>
So far this post has only discussed placing in relation to return types. For our
goal of preserving express compatibility with the existing stdlib, placing
return types are not enough. Back in March Eric Holk and Tyler Mandry argued
that we’ll also want to have some form of placing arguments</em> (or at least the
capability that enables us to do this). So let’s use Box::new</code> as our example to
show why. Without placing arguments, the best we could do would be to define
some form of Box::new_with</code> function that takes a placing closure:</p>
impl</span><T> Box<T> {
</span>    </span>// The existing default constructor
</span>    </span>fn </span>new</span>(</span>x</span>: T) -> </span>Self </span>{ ... }
</span>
</span>    </span>// The newly introduced `placing` constructor
</span>    </span>fn </span>new_with</span><F>(</span>f</span>: F) -> </span>Self
</span>    </span>where
</span>        F: #[placing] FnOnce() -> T,
</span>    { ... }
</span>}
</span></code></pre>
The new_with</code> constructor is always preferable to the new</code> constructor
because it guarantees the absence of intermediate stack copies. This will lead
to an effective deprecation Box::new</code>. If not outright, then likely first by way of ”best practices”</em>.</p>
The way to solve this would be to enable Box::new</code> to act as the receiver of
values which need to be emplaced. This would be done by requiring annotations
not at the function level, but at the argument/return-type level. Keeping with our #[placing]</code> placeholder notation, we can imagine it looking something like this:</p>
//! Original
</span>
</span>impl</span><T> Box<T> {
</span>    </span>// `Box::new` here takes type `T` and
</span>    </span>// constructs it in-place on the heap
</span>    </span>fn </span>new</span>(</span>x</span>: #[placing] T) -> </span>Self </span>{ ... }
</span>}
</span></code></pre>
I expect the desugaring of this will likely look somewhat similar to Alice
Ryhl’s in-place initialization RFC</a>,
desugaring to some form of impl Emplace</code> trait. But crucially: this would only
be observable within the implementation, and not to any of the callers.</p>
//! Desugared
</span>
</span>/// Write a value to a place.
</span>trait </span>Emplace<T> {
</span>    </span>fn </span>emplace</span>(</span>self</span>, </span>slot</span>: </span>*mut</span> T);
</span>}
</span>
</span>impl</span><T> Box<T> {
</span>    </span>fn </span>new</span>(</span>x</span>: impl Emplace<T>) -> </span>Self </span>{
</span>        </span>let mut</span> this = Box::<T>::new_uninit(); </span>// 1. create the place
</span>        x.</span>emplace</span>(this.</span>as_mut_ptr</span>());          </span>// 2. init the value
</span>        Ok(this.</span>assume_init</span>())                 </span>// 3. all done
</span>    }
</span>}
</span></code></pre>
Now the reason why I’ve put this under Q&A is because I haven’t yet figured out
the finer language rules here since I haven’t implemented this yet. As a core
design constraint: invoking a function that takes placing arguments should be no
different from a regular function. This is needed to keep APIs
backwards-compatible.</strong> What should make this special is that functions with
placing arguments and placing return types should work together to emplace.</p>
What about borrows / local lifetime extensions?</h3>
In her post on super let</a>, Mara
provides a clear example for when temporary lifetime extensions are useful. Here
Writer::new</code> takes an &'a File</code>, and we need  a feature like super let</code> to
create an instance of File</code> that outlives the scope of the block:</p>
let</span> writer = {
</span>    println!("</span>opening file...</span>");
</span>    </span>let</span> filename = "</span>hello.txt</span>";
</span>    </span>super let</span> file = File::create(filename).</span>unwrap</span>();
</span>    Writer::new(&file)
</span>};
</span></code></pre>
The return type Writer</code> here must have a lifetime to be able to reference
super let file</code>. But it can’t be a normal lifetime, since it doesn’t adhere to
the usual rules. Without specifying any of the concrete rules, this lifetime has
been dubbed 'super</code>. From the perspective of the block this behaves not unlike
'static</code> - though crucially it is not the same as 'static</code>.</p>
Now the question is: how can we represent this block as a function instead?
Because it makes sense that we would want to eventually be able to factor out
functionality from blocks into functions. We’d probably want to do that using a
'super</code> lifetime, like so:</p>
//! Original
</span>
</span>fn </span>create_writer</span>(</span>filename</span>: &</span>str</span>) -> Writer<</span>'super</span>> {
</span>    println!("</span>opening file...</span>");
</span>    </span>super let</span> file = File::create(filename).</span>unwrap</span>();
</span>    Writer::new(&file)
</span>}
</span></code></pre>
Note how Writer</code> itself does not require a #[placing]</code> annotation: it’s okay
that we copy it out of the function. The only important part is that  file</code>
outlives the current scope. The desugaring for this is quite fun, even if we
can’t yet represent it in the type system. What we need to do here is ensure
that file</code> is constructed in-place in the caller’s scope. And once initialized
we can reference it using a blank/unsafe lifetime in our return type. I haven’t
actually checked this, but I believe this is a valid desugaring:</p>
//! Desugared
</span>
</span>fn </span>create_writer</span><</span>'a</span>>(</span>filename</span>: &</span>str</span>, </span>file</span>: &</span>'a mut </span>MaybeUninit<File>) -> Writer<</span>'a</span>> {
</span>    println!("</span>opening file...</span>");
</span>    </span>let</span> file = </span>unsafe </span>{ file.</span>write</span>(File::create(filename).</span>unwrap</span>()) };
</span>    Writer::new(file)
</span>}
</span></code></pre>
This however needs to be paired with a function prelude on invocation to create
the place for file</code>. We can therefore imagine the invocation of this function
looking something like this:</p>
// with syntactic sugar
</span>let</span> writer = </span>create_writer</span>("</span>hello.text</span>");
</span>
</span>// desugared
</span>let mut</span> file = MaybeUninit::uninit();
</span>let</span> writer = </span>create_writer</span>("</span>hello.text</span>", &</span>mut</span> file);
</span></code></pre>
What about pinning?</h3>
Once we have emplacement in the language, most of the reasons for Pin</code> being
the way it is kind of fall away. But there is a gap between having emplacement,
and then also having !Move</code>, and so we do need to have some form of
compatibility with Pin</code>. Luckily the Pin</code> type is just a special-case of the
previous lifetime extension example.</p>
What’s neat about placing functions is that it would allow us to replace the std::pin::pin!</code> macro with a pin</code> free-function, using the 'super</code> lifetime:</p>
//! Original
</span>
</span>pub fn </span>pin</span><T>(</span>t</span>: T) -> Pin<&</span>'super mut</span> T> {
</span>    </span>super let mut</span> t = t;
</span>    </span>unsafe </span>{ Pin::new_uninit(&</span>mut</span> t) }
</span>}
</span></code></pre>
All the desugaring needs to do to make this work is change the function to take an additional
slot MaybeUninit<T></code> that we can write our value into. This allows us to extend
the lifetime, after which we can reference it in our return type like so:</p>
//! Desugared
</span>
</span>pub fn </span>pin</span><T, </span>'a</span>>(</span>t</span>: T, </span>slot</span>: &</span>'a mut </span>MaybeUninit<T>) -> Pin<&</span>'a mut</span> T> {
</span>    </span>let mut</span> t = </span>unsafe </span>{ slot.</span>write</span>(t) };
</span>    </span>unsafe </span>{ Pin::new_uninit(&</span>mut</span> t) }
</span>}
</span></code></pre>
Are annotations necessary?</h3>
RFC 2884: Placement by Return</a>
proposed introducing C++’s Guaranteed Copy Elision rules to Rust almost as-is. I appreciate this
RFC because it has the right idea about the scope of the changes, and does it
solely by changing the meaning of return</code>. But where it runs into trouble is
that it only</em> changes the meaning of return</code>. And so instead of adding
placement to e.g. Box::new</code>, it needs to add a new method Box::new_with</code>.</p>
Fundamentally there are three kinds of placement we’re interested in:</p>

Placing return types</strong>: where we want a to avoid copying the type returned from a function. For example if we want a referentially-stable constructor.</li>
Placing function arguments</strong>: where we want to avoid copying an argument passed into a function. For example: to when constructing a type on the heap.</li>
Lifetime extensions</strong>: where we want to reference a local variable from a local type which will outlive the current scope (lifetime extension). For example: when pinning.</li>
</ul>
Even if C++’s Guaranteed Copy Elision should be our ultimate goal; annotations
allow us to get there incrementally. Placing return types are fairly easy.
Placing function arguments are a little harder. And lifetime extensions will be
harder still. Being able to opt into this via explicit annotations means we can
start small and gradually build up.</p>
For something as broad as “functions with arguments or return types” that seems
like the right way to start. And if for some reason this feature ends up being
so successful we’ll want to annotate nearly every function with it, changing
defaults seems like something that could be done over an edition if we wanted
to.</p>
What about self-referential types?</h3>
I’ve written at length about self-referential types</a> before. Once we have placing functions, we end up with three components for generalized self-referential types are:</p>

Placing functions</strong>: so we can construct a type in a stable position in memory</li>
Self-lifetimes</strong>: so you can declare that some field borrows from some other field.</li>
Partial constructors</strong>: so you can start by initializing the owned data first, and initialize the references to that data second.</li>
</ol>
We’ve already seen placing functions. Self-lifetimes would allow fields to refer
to data contained in other fields, which would probably look something like this:</p>
struct </span>Cat {
</span>    </span>data</span>: String,
</span>    </span>name</span>: &</span>'self</span>.data </span>str</span>, </span>// ← references `self.data`
</span>}
</span></code></pre>
And partial constructors would allow you to construct types in multiple phases.
For example, here we first initialize the field data</code> in Cat</code>. And then we
take the string contained within, and do some crude parsing to interpret
everything up until the first space as the cat’s name:</p>
fn </span>new_cat</span>(</span>data</span>: String) -> Cat {
</span>    </span>let mut</span> cat = Cat { data, .. };
</span>    cat.name = cat.data.</span>split</span>(' ').</span>next</span>().</span>unwrap</span>();
</span>    cat
</span>}
</span></code></pre>
Once we have this, we can of course combine it with the lifetime extension
examples we’ve shown before to ensure that the type remains in a stable memory
location. But crucially: this is also forward-compatible with alternative
mechanisms for referential stability such as the Move</code> auto-trait.</p>
As a side note: partial constructors are basically the same feature as view
types and pattern
types</a>. It’s
still the same general refinement feature, but now with an added rule that we
can go from a refinement type back to the original type by populating its
fields. Assigment here takes the place of an inverse match</code> if you will.</p>
What’s nice about this design is that these features are all orthogonal, but
complimentary. Emplacement is useful even without partial initialization. And
partial initialization (refinement) is useful even without self-referential
lifetimes. Features complimenting each other in this way to me is the hallmark
of good language design. It means it generalizes beyond just a niche use case.
But simultaneously becomes even more useful when combined with other features.</p>
Why not directly rely on Init<T></code> or &out</code>?</h3>
Both the Init</code> type and &out</code> parameters feature are backwards-incompatible to
add to existing types and interfaces. This is a problem, because placement is
broadly applicable: we know from C++ 17 that virtually every constructor wants to be
placing. And we can’t reasonably rewrite every function returning -> T</code> to instead
return -> Init<T></code> or take &out T</code>.</p>
// 1. Original signature
</span>fn </span>new_cat</span>() -> Cat { ... }
</span>
</span>// 2. Using `Init`, changes the signature
</span>fn </span>new_cat</span>() -> Init<Cat> { ... }
</span>
</span>// 3. Using `&out`, changes the signature
</span>fn </span>new_cat</span>(</span>cat</span>: &out Cat) { ... }
</span> 
</span>// 3. Using `#[placing]`, preserves the signature
</span>#[</span>placing</span>]
</span>fn </span>new_cat</span>() -> Cat { ... }
</span></code></pre>
That doesn’t mean that these designs are inherently broken or incorrect; far
from it actually. But because they seem to assume a different scope for the
design, it naturally means those designs are operating with a different set of
design constraints - in turn leading to different designs.</p>
I believe that RFC 2884: Placement by
Return</a> by PoignardAzur had the
right idea. In order for Rust to be competitive with C++, we need to be able to
guarantee most constructors can emplace. And in order to do that, we can’t
require people to rewrite their code.</p>
However, the Init RFC</a> has
some great ideas about how to emplace function arguments. Which is something
that RFC 2884 didn’t have a good answer to. I believe that Rust’s strength lies
in its ability to distill different ideas and synthesize them into something
new. I believe that we can arrive at something truly great if we combine super let</code>, placement-by-return, Init</code>, and ensure it is backwards-compatible.</p>
Can placing functions be nested?</h3>
Yes - placing functions called in a return position should be able to compose.
For the language feature should be relatively straight-forward when calling one
placing function inside of another in the return position:</p>
struct </span>Foo {}
</span>#[</span>placing</span>]
</span>fn </span>inner</span>() -> Foo {
</span>    Foo {} </span>// ← 1. Constructed in the caller's scope
</span>}
</span>
</span>#[</span>placing</span>]
</span>fn </span>outer</span>() -> Foo {
</span>    </span>inner</span>() </span>// ← 2. Forwards the emplacement to its caller
</span>}
</span></code></pre>
This itself is reminiscent of C++’s guaranteed copy elision guarantees, which
are composable through functions. That is because in C++ temporaries are only
materialized into real objects at the end of a call chain, enabling arbitrarily
deep composition. For Rust it’s important that we maintain this same property,
including when wrapping and composing types:</p>
struct </span>Foo {}
</span>#[</span>placing</span>]
</span>fn </span>inner</span>() -> Foo {
</span>    Foo {} </span>// ← 1. Constructed in the caller's scope
</span>}
</span>
</span>struct </span>Bar(Foo)
</span>#[</span>placing</span>]
</span>fn </span>outer</span>() -> Bar {
</span>    Bar(</span>inner</span>()) </span>// ← 2. Emplaces Bar in the caller, and Foo in Bar
</span>}
</span></code></pre>
In this example Bar</code> is constructed in the caller’s scope, and as part of
initialization it invokes and emplaces Foo</code> inside of it. I stopped working on
the placing crate before implementing this, but for this desugaring to work all
we need to do is for the “place” used by the inner</code> function to by placed
inside of the “place” used by the outer</code> function.</p>
The most complex kind of composition is when we start involving temporaries.
Consider the following example which constructs a type in the first function,
mutates it in the second function, and finally uses it in a third function:</p>
/// A Cat which is constructed in place and can meow.
</span>struct </span>Cat {}
</span>impl </span>Cat {
</span>    #[</span>placing</span>]
</span>    </span>fn </span>new</span>(</span>name</span>: String) -> </span>Self </span>{ .. }
</span>    </span>fn </span>set_name</span>(&</span>mut </span>self</span>) { .. }
</span>    </span>fn </span>meow</span>(&</span>self</span>) { .. }
</span>}
</span>
</span>/// Construct a value.
</span>#[</span>placing</span>]
</span>fn </span>first</span>() -> Cat {
</span>    </span>let</span> name = "</span>Nori</span>".</span>to_string</span>();
</span>    Cat::new(name) </span>// ← 1. Emplaced in the caller
</span>}
</span>
</span>/// Mutate the value.
</span>#[</span>placing</span>]
</span>fn </span>second</span>() -> Cat {
</span>    </span>super let mut</span> cat = </span>first</span>();        </span>// ← 2. Emplaced in the caller
</span>    cat.</span>set_name</span>("</span>Chashu</span>".</span>to_string</span>()); </span>// ← 3. Mutated
</span>    cat                                 </span>// ← 4. Logically returned
</span>}
</span>
</span>/// Use the value.
</span>fn </span>third</span>() {
</span>    </span>let</span> cat = </span>second</span>(); </span>// ← 5. Placed on-stack here
</span>    cat.</span>meow</span>();
</span>}
</span></code></pre>
We need to have some way to emplace temporaries that we later logically return
from the caller. The super let</code> feature seems like it would particularly well
for this. In the 'move</code> example we were returning a reference to a super let</code>
value from a function. But I think it would make a lot of sense if we could use
super let</code> to return logically owned values from a function too, as shown here.</p>
Conclusion</h2>
This post introduces a design for placing functions: a declarative addition to
Rust that enables types to be constructed in-place. Unlike alternative APIs such
as Init<T></code> and &out</code>, placing functions are designed to keep function
signatures intact, enabling existing functions and APIs to be annotated as
retroactively.</strong> In other words: placing functions were designed to prioritize
backwards-compatibility.</p>
The placing functionality has been prototyped in the placing
crate</a>. Even though this crate was
designed with limited time and entirely using procedural macros, it should be
sufficient to prove the feasibility of a language feature.</p>
While placing functions are the most important placement-related feature, they
are not the only one. There are three kinds of placing functions total that need
to be addressed:</p>

Placing return types</strong>: where we want a to avoid copying the type returned from a function. For example if we want a referentially-stable constructor.</li>
Lifetime extensions</strong>: where we want to reference a local variable from a local type which will outlive the current scope (lifetime extension). For example: when pinning.</li>
Placing function arguments</strong>: where we want to avoid copying an argument passed into a function. For example: to when constructing a type on the heap.</li>
</ol>
A complete solution should address all three of these uses. Placing functions
only directly enables placing return types, but in the discussion section we’ve
also discussed how we could extend this to include lifetime extensions and
placing function arguments.</p>
When discussing emplacement it’s important to consider both intra-function and
inter-function variants. The experimental super let</code> feature only works
within</em> functions today (intra-function). With placing functions working very
similarly, enabling similar functionality to work between</em> functions too
(inter-function). A good design should make both variants easy, convenient, and
interoperable.</p>
In this post we’ve also explained why the scope for emplacement is broad: in C++
constructors guarantee emplacement by default. And given Rust wants to have
performance that’s competitive with C++, we have to assume that most functions
will eventually want to make that guarantee as well. And the only realistic way
to achieve that is if we can guarantee emplacement in a backwards-compatible
way.</p>
Thanks to Sy Brand for reviewing earlier copies of this post and especially providing valuable feedback and clarifications about both C++’s copy elisions feature, as well as how the SYSV ABI works.</em></p>
^{1</sup>
I only had a hunch this was possible. Alice Ryhl has done the work to prove this can be done. Albeit for a slightly different, but very similar proposal.</p>
</div>}

Tree-Structured Concurrency II: Replacing Background Tasks With Actors
Wed, 02 Jul 2025 00:00:00 +0000
Structured concurrency</h2>
(Tree-)Structured
Concurrency</a> is neat
because it greatly simplifies concurrent programs. It greatly reduces, if not
outright eliminates the possibility of logical races due to concurrency issues. And
conceptually it’s not that hard either, as we can encode structured concurrency
with just two rules:</p>

Every child computation (except the root) must</em> have exactly one logical parent at any given time during execution.</li>
Every parent computation may</em> have multiple child computations executing concurrently at any given moment.</li>
</ol>
Every child must always have exactly one parent. But every parent may have multiple children. These two rules allow us to infer a third rule that is at the core of the system:</p>

Child computations will never outlive their parents.</li>
</ol>
If we apply these rules, we’ll notice that the call-graph of a program will
naturally form a tree. This is nothing special, since non-concurrent programs
already work this way. If that’s not intuitive: realize that for example
flame graphs and flame charts are nothing more than a visualization of
tree-shaped data.</p>
The background task problem</h2>
One of the most common questions people ask when learning about structured concurrency is:</p>

“How do I run work after</em> returning from a function?”</p>
</blockquote>
This is a fair question! A common example is when implementing an HTTP server: you
may want to log metrics on every request, but you don’t want to delay sending
the response until logging has completed. At which point people often reach for
unstructured concurrency in the form of background tasks</em>, often using some
form of task::spawn</code>:</p>
/// Some HTTP handler
</span>async </span>fn </span>handler</span>(</span>req</span>: Request, </span>_context</span>: ()) -> Result<Response> {
</span>    </span>// Get the body from a request
</span>    </span>let</span> body = req.</span>body</span>().</span>try_to_string</span>()?;
</span>
</span>    </span>// Create a detached background task that
</span>    </span>// does some work in the background
</span>    task::spawn(async {
</span>        </span>background_work</span>(body).await.</span>unwrap</span>();
</span>    });
</span>
</span>    </span>// Construct a response and return
</span>    </span>// it from the handler
</span>    Ok(Response::Ok())
</span>}
</span></code></pre>
In this example, task::spawn</code> creates a background task which is not owned by
any parent. This violates the second rule of structured concurrency: every
computation should always have one logical parent. We can see this being a
problem if background_work</code> were to complete unsuccessfully. Right now if it
fails we have no way to propagate the error or retry the operation, so all we
can realistically do is let it crash.</p>
In this example we also have no way of cancelling the work done in the task
since it’s not rooted in any logical parent. But despite those issues, use case
itself is also a very reasonable one: we shouldn’t have to wait for background
work to finish before sending off a response. So: what do we do?</p>
The actor pattern</h2>
What we want is for work to happen in the background, while simultaneously also
being rooted in some parent computation. I believe the right way to solve this
is by using actors: types that we can send data and will schedule work for us.</p>
In frameworks
like actix</a> and
choo</a> this is done using messages</em> that are
dispatched using channels or event emitters. But languages like Swift have
actor
support</a>
built directly into the language, and rely on methods instead of messages.</p>
But for what we’re trying to do here we don’t really need any fancy libraries or
frameworks: we can easily build our own actors using structs, impl blocks, and
channels. Here is a basic template you can use to implement your own actors
with. Without comments this comes out to maybe 15 lines total:</p>
// Using the `async-channel` and
</span>// `futures-concurrency` libraries
</span>use </span>async_channel::{</span>self </span>as channel, Sender, Receiver};
</span>use </span>futures_concurrency::prelude::*;
</span>
</span>/// The type of data we will send to our actor.
</span>/// This is just a simple alias to our HTTP
</span>/// framework's `Body` type.
</span>type </span>Message = Body;
</span>
</span>/// Our custom actor type
</span>struct </span>Actor(Receiver<Message>);
</span>
</span>/// A handle to the actor which can be
</span>/// used to schedule messages with.
</span>type </span>Handle = Sender<Message>;
</span>
</span>impl </span>Actor {
</span>    </span>/// Create a new `Actor` instance
</span>    </span>fn </span>new</span>(</span>capacity</span>: </span>usize</span>) -> (</span>Self</span>, Handle) {
</span>        </span>let </span>(sender, receiver) = channel::bounded(capacity);
</span>        (</span>Self</span>(receiver), sender)
</span>    }
</span>
</span>    </span>/// Start listening for incoming messages
</span>    </span>/// and handle them concurrently
</span>    async </span>fn </span>run</span>(&</span>mut </span>self</span>) -> Result<()> {
</span>        </span>// Using `ConcurrentStream` from `futures-concurrency`
</span>        </span>// to take work from the channel and execute it
</span>        </span>// concurrently. To bound the amount of concurrency,
</span>        </span>// you can use the provided `.limit` combinator.
</span>        </span>self</span>.</span>0.</span>co</span>().</span>try_for_each</span>(|</span>msg</span>| {
</span>            </span>// Work is scheduled here
</span>            </span>background_work</span>(body).await?;
</span>            Ok(())
</span>        }).await?;
</span>        Ok(())
</span>    }
</span>}
</span></code></pre>
Now to use this we need to create an instance, run it concurrently with our HTTP
server, and make sure request handlers in the server have access to it. I’m most
familiar with the Tide HTTP framework</a> so
I’ll be using that here, but your favorite framework probably allows you to do
something similar. Here is the way to tie this all together:</p>
use </span>futures_concurrency::prelude::*;
</span>
</span>#[</span>async_main</span>]
</span>async </span>fn </span>main</span>() -> server::Result<()> {
</span>    </span>// Create an instance of our actor
</span>    </span>let </span>(actor, handle) = Actor::new();
</span>    
</span>    </span>// Create an instance of our server
</span>    </span>// and pass it the actor handle
</span>    </span>let mut</span> app = Server::with_context(handle);
</span>    app.</span>at</span>("</span>/submit</span>").</span>post</span>(handler);
</span>
</span>    </span>// Start both the actor and the server
</span>    </span>// and execute them concurrently
</span>    </span>let</span> a = app.</span>listen</span>("</span>127.0.0.1:8080</span>");
</span>    </span>let</span> b = actor.</span>run</span>();
</span>    (a, b).</span>try_join</span>().await?;
</span>    
</span>    Ok(())
</span>}
</span></code></pre>
And now that our server has a handle, we can change our request handler function
to schedule work on the actor rather than in a dangling task:</p>
async </span>fn </span>handler</span>(</span>req</span>: Request, </span>handle</span>: Handle) -> Result<Response> {
</span>    </span>let</span> body = req.</span>body</span>().</span>try_to_string</span>()?;
</span>    handle.</span>send</span>(body).await?; </span>// ← Send work to the actor
</span>    Ok(Response::Ok())
</span>}
</span></code></pre>
While this still relies on performing an async operation that may potentially
fail, this is still meaningfully different. Rather than waiting for the work to
have been completed</em>, all we’re now waiting for is for the work to have been
enqueued</em>. And unless something is going horribly wrong in the system, that
should happen virtually instantly. And if for some reason it doesn’t: that’s why we always add timeouts to our computations right? …right?</a></p>
How are actors different from globals?</h2>
This section was added on 2025-06-02, after publishing the post.</em></p>
I’ve had a number of people ask what the difference is between the actor pattern
shown in this post, and the global task pool that manages the spawned tasks.
After all: if you take the program as the root, both variants end up looking
tree-shaped.</p>
The difference between the two is that under structured concurrency we’re not
treating the program</em> as the root, but the main function</em> as the root. The
actor pattern in this post turns an unreachable, global task pool into a
reachable task pool with a locality that can be managed however we like. Sy
Brand put this particularly well; paraphrasing ever so slightly:</p>

[…] that "oh, it's structured as long as you look at it this way" smells like
"well my program doesn't leak memory since it will all be reclaimed when it
terminates".</p>
</blockquote>
The point of structured concurrency is that everything is always managed and
reachable by having all computation existing in the same tree. This makes it
possible to always</em> handle errors, restart instances, and shut things down gracefully.</p>
Conclusion</h2>
In this post we’ve answered one of the most common questions people have when
first encountering structured concurrency: “How do I schedule background work
without relying on dangling tasks?”</em></p>
The simplest answer to this question are actors: types that provide some
(shareable) handle that can be used to receive work on. We’ve shown how to
define an actor in about 15 lines of code, how to integrate it with an HTTP
server, and finally using it to replace an existing background task.</p>
I think of structured concurrency the way I think about memory safety: the more
we make it the default, the fewer concurrency issues people will have overall.
And that seems important because concurrent, asynchronous, and parallel
computing tends to some of the hardest code to reason about and debug.</p>
Appendix A: Actor pattern template</h2>
use </span>async_channel::{</span>self </span>as channel, Sender, Receiver};
</span>use </span>futures_concurrency::prelude::*;
</span>
</span>type </span>Message = ();
</span>type </span>Handle = Sender<Message>;
</span>
</span>struct </span>Actor(Receiver<Message>);
</span>impl </span>Actor {
</span>    </span>fn </span>new</span>(</span>capacity</span>: </span>usize</span>) -> (</span>Self</span>, Handle) {
</span>        </span>let </span>(sender, receiver) = channel::bounded(capacity);
</span>        (</span>Self</span>(receiver), sender)
</span>    }
</span>
</span>    async </span>fn </span>run</span>(&</span>mut </span>self</span>) -> Result<()> {
</span>        </span>self</span>.</span>0.</span>co</span>().</span>try_for_each</span>(|</span>msg</span>| {
</span>            todo!("</span>schedule work here</span>")
</span>        }).await
</span>    }
</span>}
</span></code></pre>

Async Traits Can Be Directly Backed By Manual Future Impls
Mon, 26 May 2025 00:00:00 +0000
Introduction</h2>
There’s a fun little tidbit that most people don’t seem to be aware of when
writing async functions in traits (AFITs) and that is: they allow you to
directly return futures from methods. Sounds confusing? Let me illustrate what I
mean with a simple example. Assume we have an a trait AsyncIterator</code> which has an async fn next</code> method that is defined as follows:</p>
trait </span>AsyncIterator {
</span>    </span>type </span>Item;
</span>    async </span>fn </span>next</span>(&</span>mut </span>self</span>) -> Option<</span>Self::</span>Item>;
</span>}
</span></code></pre>
If we wanted to define an iterator which yields an item exactly once, we would
probably write it as follows. This stores a value T</code> in an Option</code>, and in a
call to next</code> we extract it and return Some(T)</code> if we have a value, and None</code>
if we don’t:</p>
/// Yields an item exactly once
</span>pub struct </span>Once<T>(Option<T>);
</span>impl</span><T> AsyncIterator </span>for </span>Once<T> {
</span>    </span>type </span>Item = T;
</span>    async </span>fn </span>next</span>(&</span>mut </span>self</span>) -> Option<T> {
</span>        </span>self</span>.</span>0.</span>take</span>()
</span>    }
</span>}
</span></code></pre>
Easy right? Now let’s try and write this future by hand. Rust is a systems
programming language, and so it’s important that we can drop into lower levels
of abstraction to gain additional control when needed. I consider it to be a gap
in the language whenever we can’t break through an abstraction into its
constituent parts. So let’s see what that would look like.</p>
Naively backing AFITs with manual future impls</h2>
Ok, so what are the constituent parts of an async fn</code>? Well, that’s mainly the
Future</code> trait. What we want to do here is to define our own future and use
that as the basis of our implementation. All we do is deref an option, and take
its internal value - which makes it rather simple. Naively we would probably
write something like this:</p>
/// The internal `Future` impl of our `Once` iterator
</span>struct </span>OnceFuture<</span>'a</span>, T>(&</span>'a mut </span>Once<T>);
</span>impl</span><</span>'a</span>, T> Future </span>for </span>Once<</span>'a</span>, T> {
</span>    </span>type </span>Output = T;
</span>    </span>fn </span>poll</span>(</span>self</span>: Pin<&</span>mut Self</span>>, </span>cx</span>: &</span>mut </span>Context<</span>'a</span>>) -> Poll<</span>Self::</span>Output> {
</span>        </span>// SAFETY: we're projecting into an unpinned field
</span>        </span>let</span> this = </span>unsafe </span>{ Pin::into_inner_unchecked(</span>self</span>) };
</span>        Poll::Ready((&</span>mut</span> this.</span>0.0</span>).</span>take</span>())        
</span>    }
</span>}
</span>
</span>/// Yields an item exactly once
</span>pub struct </span>Once<T>(Option<T>);
</span>impl</span><T> AsyncIterator </span>for </span>Once<T> {
</span>    </span>type </span>Item = T;
</span>    async </span>fn </span>next</span>(&</span>mut </span>self</span>) -> Option<T> {
</span>        </span>// Delegate to the `OnceFuture` impl
</span>        OnceFuture(</span>self</span>).await
</span>    }
</span>}
</span></code></pre>
There’s a lot going on all of a sudden, right? This is the low-level version of
what we wrote earlier. Or is it? If you look carefully you might notice that
we’re dealing with one extra level of indirection here. In our iterator’s main
body we wrote: OnceFuture(self).await</code>.  In a simple benchmark the compiler
will happily optimize this. But in more complex programs the compiler may not.
And that’s an issue, because it means that switching to a lower-level
abstraction may yield worse performance.</p>
If this was the best we could do, it would likely spell the death of AFITs in
the stdlib. It would mean that AFITs would be useful for high-level APIs that we
can implement using async fn</code> and be happy with that. But not for serious</em>
implementations which need to be optimized all the way, like those found in the
stdlib. And that would point us at things like poll_*</code>-based traits which we
can more directly implement by hand.</p>
Directly backing AFITs with manual future impls</h2>
Luckily we can trivially remove the intermediate .await</code> call from our previous
example using a little-known AFIT feature: the ability to directly return
futures. Instead of writing async fn next</code>, we can write that same function as
fn next() -> impl Future</code>. That’s almost the same; with some minor differences:</p>
/// The internal `Future` impl of our `Once` iterator
</span>struct </span>OnceFuture<</span>'a</span>, T>(&</span>'a mut </span>Once<T>);
</span>impl</span><</span>'a</span>, T> Future </span>for </span>Once<</span>'a</span>, T> {
</span>    </span>type </span>Output = T;
</span>    </span>fn </span>poll</span>(</span>self</span>: Pin<&</span>mut Self</span>>, </span>cx</span>: &</span>mut </span>Context<</span>'a</span>>) -> Poll<</span>Self::</span>Output> {
</span>        </span>// SAFETY: we're projecting into an unpinned field
</span>        </span>let</span> this = </span>unsafe </span>{ Pin::into_inner_unchecked(</span>self</span>) };
</span>        Poll::Ready((&</span>mut</span> this.</span>0.0</span>).</span>take</span>())        
</span>    }
</span>}
</span>
</span>/// Yields an item exactly once
</span>pub struct </span>Once<T>(Option<T>);
</span>impl</span><T> AsyncIterator </span>for </span>Once<T> {
</span>    </span>type </span>Item = T;
</span>    </span>// Directly returns the `OnceFuture` impl
</span>    </span>fn </span>next</span>(&</span>mut </span>self</span>) -> impl Future<Output = Option<T>> {
</span>        OnceFuture(</span>self</span>) </span>// ← no more `.await`
</span>    }
</span>}
</span></code></pre>
Do you see the fn next</code> impl? It now directly returns the OnceFuture</code>. And we
can do this even though the trait definition explicitly defined the trait method
as async fn next</code>. This means there is no intermediate .await</code> call required
to base our AFIT-based trait on a manual future impl. Which means we’re no longer
depending on the compiler to optimize this for us to have equivalent
performance, meeting our goals of being able to manually lower the high-level
notation into its constituent parts which we can manually control.</p>
Simple inline poll state machines</h2>
The Rust stdlib provides a future::poll_fn</code> convenience function that allows
you to create stateless (e.g. does not hold its own state across poll</code> calls)
futures. But because it takes an FnMut</code> closure, it can reference external
state made available to it via upvars. This allows us to rewrite the chonky manual future implementation from our last two examples as a sleek inline call to poll_fn</code>:</p>
/// Yields an item exactly once
</span>pub struct </span>Once<T>(Option<T>);
</span>impl</span><T> AsyncIterator </span>for </span>Once<T> {
</span>    </span>type </span>Item = T;
</span>    </span>fn </span>next</span>(&</span>mut </span>self</span>) -> impl Future<Output = Option<T>> {
</span>        future::poll_fn(|</span>_cx</span>| </span>/* -> Poll<Option<T>> */ </span>{
</span>            Poll::Ready((&</span>mut </span>self</span>.value).</span>take</span>())
</span>        })
</span>    }
</span>}
</span></code></pre>
This is incredibly convenient because it allows us to quickly write some
optimized inline poll-based state machine code inside trait implementations. Or
really: any async</code> context. I like to think of poll_fn</code> as a way to quickly
step into “low level async” mode, kind of like how unsafe {}</code> allows you to
step into “raw memory mode”.</p>
This also clearly makes the Future</code> trait the fundamental building block of
everything async</code>. Which is a simpler and more robust model than the
alternative fn poll_*</code> model; under which async operations can be implemented
either in terms of Future::poll</code>, AsyncIterator::poll_next</code>,
AsyncRead::poll_read</code>, and so on.</p>
Conclusion</h2>
I wrote this post because most people don’t seem to be aware that AFITs can be
used like this. Though arguably it’s one of its most important capabilities. And
that’s important because as I said earlier in this post: it would be really bad
if the async trait space was bifurcated between:</p>

AFIT-based traits</strong>: which are convenient to implement but have worse performance due to lack of control.</li>
Poll-based traits</strong>: which are inconvenient to implement but have better performance because of the control provided</li>
</ul>
This post shows that AFIT-based traits are both convenient to implement, and
when desired provide the control needed to guarantee performance via manual
poll-state machine implementations.</strong> This unifies the design space for
async traits, removing the choice between “the easy API” and “the fast API”.
With AFITs the easy API is</em> the fast API.</p>
While we used the AsyncIterator</code> trait as an example in this post, nothing in
this post so far has been specific to AsyncIterator</code>. Being able to control the
future state machines of AsyncRead</code>, AsyncWrite</code>, and so on isn’t any less
important. But if we were to consider an AFIT-based AsyncIterator</code> specifically
with an async gen {}</code> feature we can see that it provides three total levels of
control, where a poll</code>-based AsyncIterator</code> trait only provides two ^{1</a></sup>:</p>}
</th> AFIT-based</th> poll_*</code>-based</th></tr></thead>

async gen {}</code></td> ✅</td> ✅</td></tr>
async fn</code></td> ✅</td> ❌</td></tr>
fn poll</code> state machine</td> ✅</td> ✅</td></tr>
</tbody></table>
And all three levels of abstraction are important to be convenient to implement.
It would be wrong to pick and choose. Being able to implement async traits based
on async fn</code> is important in cases where a high-level construct like gen {}</code>
is not available, like with AsyncRead</code>, and AsyncWrite</code>. But it’s also
important for AsyncIterator</code> to be able to implement subtrait
variants like ExactSizeIterator</code>, and be able to implement provided methods like fn size_hint</code>.
async gen {}</code> doesn’t provide the ability to do either ^{2</a></sup>, and users shouldn’t be forced to write
poll</code>-based state machines just for that.</p>}
Acknowledgements</h2>
I’d like to thank Oli Scherer for taking the time late last year to
work with me through a series of examples of AFITs backed manually implemented
futures, and explaining how they are resolved and optimized by the compiler.
That conversation is what has given me the confidence to make the claims in this
post with a degree of certainty I would otherwise not have. Oli did however not
review this post in advance, so I’m most definitely not speaking on their
behalf and any mistakes in this post will be my own.</p>
^{1</sup>
I’m sure with enough language magic we could provide some</em> way to allow fn poll_*</code>-based traits to be implemented in terms of async fn</code>. But that require an entirely new language feature, all to just end up with a less consistent and more complicated model than if we directly base all async traits (but Future</code>) on async fn</code>.</p>
</div>}^{2</sup>
Capability-based arguments aside, I also believe that implementing traits should</em> be easy. Like, what do you mean implementing a non-blocking iterator by hand requires a PhD in Rustology? Rust is supposed to make systems programming accessible and understandable</em>. And that needs to apply to every level of abstraction.</p>
</div>}
The Waker Allocation Problem
Fri, 23 May 2025 00:00:00 +0000
Introduction</h2>
The entire point of futures and Rust’s async/.await system is to introduce two
new capabilities: arbitrary concurrent execution of computations, and arbitrary
cancellation of computations. The difference between blocking and non-blocking
computation is unimportant if we then don’t also capitalize on it by scheduling
work concurrently.</p>
But there’s a problem - In order to schedule work concurrently futures have two
bad choices in how to organize their internals:</p>

Without allocations</strong> futures have O(N^2)</code>(quadratic) wakeup behavior for their child futures. This means that if we schedule 1_000 futures that all wake independently and in sequence, we’ll approximate 1_000_000 total wakes 1</a></sup>.</li>
With allocations</strong> futures can have O(N)</code> wakeup behavior for their child futures. This means that if we schedule 1_000 futures that all complete independently and in sequence, we’ll have 1_000 total wakes.</li>
</ul>
In Rust we’re well aware that hidden allocations are bad. But we also know that
quadratic scaling on potentially unbounded numbers of items is also bad. And
it’s bad that we’re in a position where we have to choose between both.</p>
Efficient concurrent execution requires intermediate wakers</h2>
Say we’re writing a manual future::join</code> function  which takes an array of impl Future</code> and returns an array of Future::Output</code>. We can model the internal state
of this using an array of futures, and an array of future outputs:</p>
/// A future that concurrently awaits a list of futures
</span>struct </span>Join<Fut: Future, </span>const</span> N: </span>usize</span>> {
</span>    </span>/// The futures we're awaiting.
</span>    </span>futures</span>: [Fut; N],
</span>    </span>/// The outputs of the futures.
</span>    </span>outputs</span>: [Option<</span>Fut::</span>Output>; N],
</span>}
</span></code></pre>
In order to concurrently await all futures, we have to implement the Future</code>
trait for Join</code>. The output type of this is [Future::Output; N]</code>, with each
call to poll</code> iterating over each pending future contained within. Here’s an
naive example implementation that, mostly just to provide context:</p>
impl</span><Fut: Future, </span>const</span> N: </span>usize</span>> Future </span>for </span>Join<Fut, N> {
</span>    </span>type </span>Output = [Fut::Output; N];
</span>
</span>    </span>fn </span>poll</span>(</span>self</span>: Pin<&</span>mut Self</span>>, </span>cx</span>: &</span>mut </span>Context<'_>) -> Poll<</span>Self::</span>Output> {
</span>        </span>// setup the initial state
</span>        </span>let</span> this = </span>unsafe </span>{ </span>self</span>.</span>get_unchecked_mut</span>() };
</span>        </span>let mut</span> all_done = </span>true</span>;
</span>
</span>        </span>// Iterate over every future until we either have all
</span>        </span>// values, or we still have futures pending.
</span>        </span>for</span> i in </span>0</span>..this.futures.</span>len</span>() {
</span>            </span>// Either the future is already done…
</span>            </span>if</span> this.outputs[i].</span>is_some</span>() {
</span>                </span>continue</span>;
</span>            }
</span>
</span>            </span>// …or we still need to poll it.
</span>            </span>let</span> fut = </span>unsafe </span>{ Pin::new_unchecked(&</span>mut</span> this.futures[i]) };
</span>            </span>match</span> fut.</span>poll</span>(cx) {
</span>                Poll::Ready(output) => this.outputs[i] = Some(output),
</span>                Poll::Pending => all_done = </span>false</span>,
</span>            }
</span>        }
</span>
</span>        </span>// The loop is done and its time to look at our
</span>        </span>// results. We either still have futures pending…
</span>        </span>if </span>!all_done {
</span>            </span>return </span>Poll::Pending;
</span>        }
</span>
</span>        </span>// …or we're all done and we're ready to return.
</span>        </span>let</span> outputs = std::mem::replace(&</span>mut</span> this.outputs, [None; N])
</span>            .</span>map</span>(Option::unwrap);
</span>        
</span>        Poll::Ready(outputs)
</span>    }
</span>}
</span></code></pre>
If you look at the core of this loop, you’ll notice that we’re iterating over
each future on every iteration - regardless of whether it called its respective
waker of not. The only futures we don't call are the ones we already have an
output of. Practically this means that we approach O(N^2)</code> execution: we wake
every future on every call to poll. And we do this because we don’t know which
futures were are ready and which are not.</p>
The solution here is to create an “intermediate” or “inline” waker. Rather than
passing the caller’s waker down into all of the child futures, the Join</code> future
should instead create its own waker(s) which can track which child futures are
ready - and only poll those. The simplest way of doing this is keeping an array of
wakers and an array of “ready” indexes. Whenever a child future calls wake, it
pushes its index to the list of ready indexes, and then calls the parent waker.
Then when the Join</code> future is polled, all it needs to do is iterate over all
ready indexes and poll those futures. For brevity I won’t bother showing the
full implementation, just the definition itself:</p>
/// A future that concurrently awaits a list of futures
</span>struct </span>Join<Fut: Future, </span>const</span> N: </span>usize</span>> {
</span>    </span>/// The futures we're awaiting.
</span>    </span>futures</span>: [Fut; N],
</span>    </span>/// The outputs of the futures.
</span>    </span>outputs</span>: [Option<</span>Fut::</span>Output>; N],
</span>    </span>/// Keeps a waker per child future
</span>    </span>wakers</span>: [ChildWaker; N],
</span>    </span>/// Tracks which child futures should be woken
</span>    </span>indexes</span>: Arc<[AtomicBool; </span>N</span>]>,
</span>}
</span>
</span>/// A waker associated with a specific child future
</span>struct </span>ChildWaker {
</span>    </span>/// The index of the child future in the index list
</span>    </span>index</span>: </span>usize</span>,
</span>    </span>/// A handle to the parent waker, passed to the struct
</span>    </span>parent_waker</span>: Waker,
</span>    </span>/// Tracks which child futures should be woken. Intended
</span>    </span>/// to be indexed into using `index` inside `Wake`.
</span>    </span>indexes</span>: Arc<[AtomicBool; </span>N</span>]>,
</span>}
</span>impl </span>Wake </span>for </span>InlineWaker { ... }
</span></code></pre>
There are more efficient schemes than this one, though this is among the
simplest. And crucially: this allows us to only ever wake the futures that need
to be woken. Which is important, since waking futures may potentially be
expensive. And while we can encourage futures to be cheap to wake spuriously -
we should really just not cause spurious wakes in the first place. Because it's
better if the burden falls on async experts writing primitives, than on every
single person implementing the future trait 2</a></sup>.</p>
Intermediate wakers require allocations</h2>
Now that we’ve looked at why futures want to make use of intermediate wakers,
let’s take a closer look at the waker internals. The Future</code> trait relies on a
type called Waker</code>, which you can think of as an Arc<Fn></code> pointer. Where you
can think of Arc</code> as a fancy type of Box<T></code> that can be freely cloned around
and provides shared (immutable) access to its internals. Here’s how you define a
ew Waker</code> that will unpark the current thread when called:</p>
use </span>std::sync::Arc;
</span>use </span>std::task::{Context, Wake};
</span>use </span>std::thread::{</span>self</span>, Thread};
</span>use </span>std::pin::pin;
</span>
</span>/// A waker that wakes up the current thread when called.
</span>struct </span>ThreadWaker(Thread);
</span>
</span>impl </span>Wake </span>for </span>ThreadWaker {
</span>    </span>fn </span>wake</span>(</span>self</span>: Arc<</span>Self</span>>) {
</span>        </span>self</span>.</span>0.</span>unpark</span>();
</span>    }
</span>}
</span></code></pre>
And here is how you construct an instance of that same waker. This relies on a
From<Arc<Wake + Send + Sync + 'static>></code>
impl</a>
for `Waker:</p>
/// Run a future to completion on the current thread.
</span>fn </span>block_on</span><T>(</span>fut</span>: impl Future<Output = T>) -> T {
</span>    </span>let mut</span> fut = pin!(fut);
</span>
</span>    </span>// Construct an instance of `ThreadWaker`
</span>    </span>let</span> t = thread::current();
</span>    </span>let</span> waker = Arc::new(ThreadWaker(t)).</span>into</span>(); </span>// ← This allocates
</span>
</span>    </span>// ...
</span></code></pre>
Right there, creating an instance of Arc::new</code> means we’re allocating. In
theory we don’t have</em> to use Arc</code> for this either. The type
Waker</code></a> only requires
that it wraps a
RawWakerVTable</code></a>
in a way that is Send + Sync + Clone</code>. That means in theory we could also have
a static</code> somewhere with internal mutability, and perhaps some (embedded)
systems rely on this for their global runtimes. But in practice for inline
concurrent operations, if we’re going to be creating a Waker</code> it’s going to
need to be based on an Arc</code>.</p>
Conclusion</h2>
The fact that it practically relies on allocations is not the only problem: in
single-threaded systems Waker</code> still requires the + Send + Sync</code> bounds. Even
if we would end up adding a
LocalWaker</code></a> type,
that still requires the original Waker</code> type to be present. Which means we
can’t practically write portable async-based !Send</code> libraries, because that
bound will never surface in the type system 3</a></sup>.</p>
The reason why we have an Arc<Fn></code> to begin with is because the future design assumes we
might want to wrap arbitrary computation in the wakers. But I’m not sure how true that is. For example, in the
futures-concurrency</code> library - our wakers only ever perform an
operation equivalent to flipping an atomic bool
(src</a>).
If we were hand-rolling our own implementation directly against e.g. the epoll</code>
API, we wouldn’t have to pay any cost like this. What we’ve looked at in this
post is purely an artifact of the Future</code> trait’s design.</p>
Having to choose between algorithmic complexity and allocations is also the
reason why I haven't proposed to upstream futures-concurrency APIs to the stdlib
yet. The Rust stdlib has a design principle that disallows hidden allocations.
But at work we also know we need to guard against API misuse. That means we’re
likely going to hit cases where using the stdlib APIs are actually not</em>
recommended. And it’s hard to justify upstreaming something we might immediately
want to guard against using.</p>
This is also not the only problem with the Future</code> trait design. I’ve spoken at
length about the problems the related Pin</code> API has. And in this post I've also
touched on the !Send</code> Waker problem. By my count the Future</code> trait has seven
problems like this, which I’ll try to describe in future posts.</p>
^{2</sup>
This is not just a hypothetical - its an issue we’ve had at Microsoft. We can’t reasonably expect that every person implementing the future trait is an async expert. And so we need primitives which don’t just pass responsibility for performance to individual users.</p>
</div>
^{1</sup>
To be fair - probably about half that, since futures which have completed aren’t polled again.</p>
</div>
^{3</sup>
I hope I don’t have to explain that having Send</code> expressed in the type system is a good thing.</p>
</div>}}}

Open and Closed Effect Systems
Thu, 22 May 2025 00:00:00 +0000
When discussing effect systems in the context of production programming
languages, I believe that the most important distinction is whether effects are
purely defined by the language, or whether users are also able to define them.
When Magnus Madsen from the Flix programming language</a> joined
the Rust T-Lang call for two sessions last December, he used the terms “open”
and “closed” to describe this distinction:</p>

Closed effect systems</strong>: are systems where all effects are all defined by
the language as built-ins. The number of effects may grow over time, but effects
are only ever defined by the language itself. This makes the number of possible
effects bounded</em>.</li>
Open effect systems</strong>: allow users of the language to also define their own
control-flow effects, which exist alongside the language built-in effects. This
makes the number of possible effects unbounded</em>.</li>
</ul>
The most interesting effects in programming languages have</em> to be built-in.
This includes things like controlling whether a function guarantees it will
terminate (“divergence”). Or whether a function is allowed to perform side
effects (“purity”).</p>
This is different from user-defined effects. These are typically defined using
“algebraic effect handlers”, but I’ve also seen the term “typed continuation”
used 1</a></sup>. I think of these more or less as language-level support for
thoroughly typed iterators/generators/coroutines. It doesn’t really make sense
to express “divergence” using iterators. Which is why even though all effect
handlers are effects, not all effects are expressed using effect handlers.</p>
In the context of Rust all we’re discussing right now is the introduction of a
closed</em> effect system. And that’s not because we don’t think an open system
would not be useful or good. But because formalizing and generalizing Rust’s
effect system in a backwards-compatible way is an incredible amount of work
already. And in order to ship, we have to draw the line somewhere 2</a></sup>.</p>
^{1</sup>
I believe there is a difference between the two terms, but I don’t quite remember. I find “typed continuation” to be a lot more evocative though. Coming from Node.js 0.8, I think of this kind of as “what if all callbacks were fully typed and we had a compiler that ensured they were handled”. Maybe not syntactically the same, but I think it’s semantically quite close.</p>
</div>
^{2</sup>
Though that doesn’t mean we can’t keep syntactic space open to consider it later. Just because we’re saying “not now” doesn’t mean we’re saying “not ever”.</p>
</div>}}

Automatic interleaving of high-level concurrent operations
Mon, 05 May 2025 00:00:00 +0000
Introduction</h2>
When working with low-level concurrency (atomics), programming languages are
generally quite eager for compilers to reorder operations if it leads to better
performance. Information about whether it's ok to reorder operations is encoded
using Atomic
Orderings</a>,
Fences</a>, and
Operations</a>. It's strange
that most programming languages that support semantic-aware reordering of
low-level concurrent operations, don't also include similar support for reordering the execution of high-level concurrent operations.</p>
To my knowledge this is true for most languages, with the notable exception of
Swift and its async let</code> construct. This feature preserves the linear-looking
nature of async code, but allows the compiler to inspect the control flow graph
and schedule operations concurrently where possible. That means that just like
with atomics, an operation that is defined later in a piece of code may finish
execution before an operation that appears earlier. Here is an example Swift
program where everything that can</em> be concurrent actually is</em> concurrent:</p>
func </span>makeDinner() async throws -> Meal {
</span>  async </span>let</span> veggies = chopVegetables()                    </span>// 1. concurrent with: 2, 3
</span>  async </span>let</span> tofu = marinateTofu()                         </span>// 2. concurrent with: 1, 3
</span>  async </span>let</span> oven = preheatOven(temperature: </span>350</span>)          </span>// 3. concurrent with: 1, 2, 4
</span>
</span>  </span>let</span> dish = Dish(ingredients: await [</span>try</span> veggies, tofu]) </span>// 4. depends on: 1, 2, concurrent with: 3
</span>  </span>return</span> await oven.cook(dish, duration: .hours(</span>3</span>))       </span>// 5. depends on: 3, 4, not concurrent
</span>}
</span></code></pre>
The Problem</h2>
To me this represents the pinnacle of language-level support for
asynchronous/concurrent programming. It makes it trivial to change any code that
may</em> be run concurrently to actually run concurrently. It enables the compiler
to take care of what is otherwise tedious and/or illegible. Take for example
this code that's written in a serial fashion using async/.await:</p>
async </span>fn </span>make_dinner</span>() -> SomeResult<Meal> {
</span>    </span>let</span> veggies = </span>chop_vegetables</span>().await?;
</span>    </span>let</span> tofu = </span>marinate_tofu</span>().await?;
</span>    </span>let</span> oven = </span>preheat_oven</span>(</span>350</span>).await;
</span>
</span>    </span>let</span> dish = Dish(&[veggies, tofu]).await;
</span>    oven.</span>cook</span>(dish, Duration::from_mins(</span>3 </span>* </span>60</span>)).await
</span>}
</span></code></pre>
Using Future::join</code> operations we can rewrite it to execute independent
operations concurrently. But this comes with the downside that the code is now
significantly less legible. Here is the same code, written using Future::try_join</code>:</p>
use </span>futures_concurrency::prelude::*;
</span>
</span>async </span>fn </span>make_dinner</span>() -> SomeResult<Meal> {
</span>    </span>let</span> dish_fut = {
</span>        </span>let</span> veggies_fut = </span>chop_vegetables</span>();
</span>        </span>let</span> tofu_fut = </span>marinate_tofu</span>();
</span>        </span>let </span>(veggies, tofu) = (veggies_fut, tofu_fut).</span>try_join</span>().await?;
</span>        Dish::new(&[veggies, tofu]).await
</span>    };
</span>    </span>let</span> oven_fut = </span>preheat_oven</span>(</span>350</span>);
</span>    </span>let </span>(dish, oven) = (dish_fut, oven_fut).</span>try_join</span>().await?;
</span>    oven.</span>cook</span>(dish, Duration::from_mins(</span>3 </span>* </span>60</span>)).await
</span>}
</span></code></pre>
To capitalize on one of the core features of async/.await</code> (ad-hoc concurrent
scheduling</em>), we had to sacrifice one of its main benefits (legibility).</strong> It's
not good when two core parts of the same feature are in tension with each other
like that. And we can't just wave a wand and tell the compiler to automatically
execute these futures concurrently. Futures tend to express operations that
change program state in one way or another. That is to say: most futures encode
side-effects</em>. And the compiler can't automatically infer which side effects
can be executed serially and which can be executed concurrently. That's because
it's not aware of the program semantics</em>.</p>
The Solution</h2>
The solution is to allow programmers to provide opt-in to explicit reorderings
in their code, just like Swift does using async let</code>. We could use a concise
notation along the lines of .co.await</code> (this is a strawman, pick your own
favorite notation). We want the notation to be in postfix position because
unlike Swift we don't want to eagerly start executing when operations are
defined, but only affect the way operations are scheduled when .await</code>ed. And
this way we never have to actually have to represent it in the type system
either 1</a></sup>. This would look something like this:</p>
^{1</sup>
Writing .co</code> without following it with an .await</code> should be a compiler error. The .co</code> would serve as a modifier on the .await</code>. Though perhaps something like C++'s co_await</code> notation is simpler. Whatever the syntax though, I don't think it should ever surface to the type system.</p>
</div>}async </span>fn </span>make_dinner</span>() -> SomeResult<Meal> {
</span>    </span>let</span> veggies = </span>chop_vegetables</span>().co.await?;
</span>    </span>let</span> tofu = </span>marinate_tofu</span>().co.await?;
</span>    </span>let</span> oven = </span>preheat_oven</span>(</span>350</span>).co.await;
</span>    
</span>    </span>let</span> dish = Dish(&[veggies, tofu]).co.await;
</span>    oven.</span>cook</span>(dish, Duration::from_mins(</span>3 </span>* </span>60</span>)).await
</span>}
</span></code></pre>
This code would directly lower to the equivalent of the Future::join</code>-based
scheme. But with the benefit of needing far less ceremony to encode the same
semantics. The other benefit of this scheme is that we always retain the option
to schedule these operations serially if we choose to. That makes this scheme
compatible with async-polymorphic functions, where manual Future::join</code> calls
are not.</p>
A feature along these lines is important, because in order to make full use of
async Rust's concurrent scheduling capabilities, any operations which can be
executed concurrently should be executed concurrently. But without language
support that comes at a severe cost to legibility, and in turn maintainability.
The only way out of this dilemma is to do what Swift has done and directly
include language support for it.</p>
Further Reading</h2>

Tasks are the Wrong Abstraction</a> - Introduces the ParallelFuture</code> trait</a>, which can be combined with the scheme outlined in this post to automatically schedule futures on multiple cores.</li>
Tree-Structured Concurrency</a> - Discusses what structured concurrency is, how to reason about it, and what's missing for Rust to fully support it.</li>
Extending Rust's Effect System</a> - Discusses among other things async-polymorphic functions.</li>
</ul>

Syntactic musings on match expressions
Tue, 29 Apr 2025 00:00:00 +0000
Introduction</h2>
One of the things that stands out to me is how similar match</code> and if..else</code>
are semantically</em>, while being diverging a fair bit syntactically</em>. The reasons for that seem mostly accidental, and I have a sneaking suspicion we could make things a little easier by making both constructs look more similar.</p>
In this post I'll be showing some control flow examples based on a queue that can either be online of offline. This is defined by an enum QueueState</code> which stores the length of the queue as a u32:</p>
enum </span>QueueState {
</span>    Online(</span>u32</span>),
</span>    Offline(</span>u32</span>),
</span>}
</span></code></pre>
Depending on whether the queue is actively having new data added to it (online),
we will want to reason differently about the queue's limits (u32</code>).  We also want to know
change the behavior based on whether the queue is full or not, and in order to
do that we will want to call a function:</p>
fn </span>is_full</span>(</span>n</span>: </span>u32</span>) -> </span>bool </span>{ .. }
</span></code></pre>
Logical AND</h2>
Take the humble if</code>-guard. We can use this in match</code> arms to chain
conditions, which is useful when we want to do something with a value contained
within a structure. Here the is_full</code> method returns a bool. But in order to
call it we have first access the number contained in the Online</code> variant:</p>
match</span> state {
</span>    QueueState::Online(count) </span>if </span>!</span>is_full</span>(count) => {
</span>    </span>//                        ^^^^^^^^^^^^^^^^^^
</span>    </span>//                            if-guard
</span>        println!("</span>queue is online and not full</span>");
</span>    }
</span>    _ => println!("</span>queue is in an unknown state</span>"),
</span>}
</span></code></pre>
if</code>-guards are really just a logical AND. In match statements we're in the
process of adding if let</code>-chaining to Rust by way of RFC 2497</a> (stable in Rust 1.88). With that we'll be able to write this same match</code> condition as follows:</p>
if let </span>QueueState::Online(count) = state && !</span>is_full</span>(count) {
</span>//                                       ^^^^^^^^^^^^^^^^^^
</span>//                                           && chaining
</span>    println!("</span>queue is online and not full</span>");
</span>} </span>else </span>{
</span>    println!("</span>queue is in an unknown state</span>");
</span>}
</span></code></pre>
This is basically the same feature exposed two different ways. Instead I really
would like to be able to use &&</code> in both cases, bringing the two closer together:</p>
match</span> state {
</span>    QueueState::Online(count) && !</span>is_full</span>(count) => {
</span>    </span>//                        ^^^^^^^^^^^^^^^^^^
</span>    </span>//                            && chaining
</span>        println!("</span>queue is online and not full</span>");
</span>    }
</span>    _ => println!("</span>queue is in an unknown state</span>"),
</span>}
</span></code></pre>
I can guarantee that just about everyone who has had exposure to a language
which C-like notation (which rounds to all programmers) will have little trouble
understanding how this works. if</code>-guards, in comparison, tend to be harder to
intuit.</p>
Logical OR</h2>
RFC 3637</a>
introduces the "guard patterns" feature: an extension to Rust's pattern notation
which that moves if</code> guards out of match expressions and into all patterns. What it functionally provides us with is a way
to chain boolean AND expressions with logical OR expressions in all patterns. Say we took our previous example, and we wanted to print
a message if is_full</code> returned false</code>, regardless of the whether the queue was
online or offline. Using chained if..let</code>s we would write this as follows:</p>
if let </span>QueueState::Online(count) = state && !</span>is_full</span>(count) {
</span>    println!("</span>queue is online and not full</span>");
</span>} </span>else if let </span>QueueState::Online(count) = state && </span>is_full</span>(count) {
</span>    println!("</span>queue is full</span>"); </span>// 1. This statement...
</span>} </span>else if let </span>QueueState::Offline(count) = state && </span>is_full</span>(count) {
</span>    println!("</span>queue is full</span>"); </span>// 2. ...is duplicated here.
</span>} </span>else </span>{
</span>    println!("</span>queue is in an unknown state</span>");
</span>}
</span></code></pre>
This is a little annoying because ideally we'd want both the second and third condition to be one big condition, separated by a logical OR</code>. Guard patterns are the feature which will allow us to do exactly that by bringing if</code>-guards and pattern OR-based pattern chaining to expressions. With that we would rewrite this as follows:</p>
if let </span>QueueState::Online(count) </span>if </span>!</span>is_full</span>(count) = state {
</span>    </span>//                           ^^^^^^^^^^^^^^^^^^
</span>    </span>//                           now using an if-guard, for fun
</span>    println!("</span>queue is online and not full</span>");
</span>} </span>else if let </span>((QueueState::Online(count) </span>if </span>is_full</span>(count) = state) | </span>// logical OR
</span>    </span>//        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
</span>    </span>//                   Either matches on pattern 1...
</span>    (QueueState::Offline(count) </span>if </span>is_full</span>(count) = state)
</span>    </span>// ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
</span>    </span>//                   ...or on pattern 2.
</span>{
</span>    println!("</span>queue is full</span>"); </span>// No more duplication!
</span>} </span>else </span>{
</span>    println!("</span>queue is in an unknown state</span>");
</span>}
</span></code></pre>
There's a lot going on here. For one, this makes if</code>-guards legal in if..let</code>
expressions. And it also allows multiple patterns to be placed used in a single
conditional using OR semantics, as long as they're appropriately parenthesized.
Let's keep going on, and use this with a match</code> expression:</p>
match</span> state {
</span>    QueueState::Online(count) </span>if </span>!</span>is_full</span>(count) => {
</span>        println!("</span>queue is online and not full</span>");
</span>    }
</span>    ((QueueState::Online(count) </span>if </span>is_full</span>(count)) |
</span>    (QueueState::Offline(count) </span>if </span>is_full</span>(count)) => {
</span>        println!("</span>queue is full</span>"); </span>// No more duplication!
</span>    }
</span>    _ => println!("</span>queue is in an unknown state</span>"),
</span>}
</span></code></pre>
This is a lot better since the flow is clearer. By not having the explicit = state</code> repeated on every line we can now actually read all the statements
left-to-right.  That's all good. We like that. Porting from the if..let</code>
version to the match</code> version was also really easy. We didn't have to change
the if..let</code> conditions at all, aside from trimming their ends. That's good!</p>
What's less good however is that we achieved this by making if..else</code>
expressions more involved. The interaction of RFC 2497 if..let</code> chains and RFC
3637 guard patterns seems particularly like it might catch people off guard.
Consider for example this line:</p>
if let </span>Foo::Bar(x) </span>if </span>cond</span>(x) = </span>bar</span>() && </span>let </span>Bin::Baz = </span>baz</span>() { .. }
</span></code></pre>
The order in which expressions are evaluated on this line is here is
2-3-1-5-4</code>. That means it's probably not advisable to mix both features. The
reason why these interactions are this way is because both RFCs operate from
fundamentally opposing premises:</strong></p>

RFC 2497</strong> believes we should enable boolean operators like &&</code> to work in more places.</li>
RFC 3637</strong> believes we should enable match-specific operators like if</code>-guards work in more places.</li>
</ul>
In the previous section we already looked at replacing if</code>-guards with the &&</code>
operator, inspired by if..let</code>-chaining. It's not a far reach to imagine if..let</code>-chaining using the ||</code> operator as well. That would look something like this:</p>
if let </span>QueueState::Online(count) = state && !</span>is_full</span>(count) {
</span>    </span>//                                   ^^^^^^^^^^^^^^^^^^
</span>    </span>//                                   if-let chain
</span>    println!("</span>queue is online and not full</span>");
</span>} </span>else if let </span>QueueState::Online(count) = state && </span>is_full</span>(count) ||
</span>    </span>//        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
</span>    </span>//                   Either matches on pattern 1...
</span>    </span>let </span>QueueState::Offline(count) = state && </span>is_full</span>(count)
</span>    </span>//  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
</span>    </span>//                   ...or on pattern 2.
</span>{
</span>    println!("</span>queue is full</span>"); </span>// No more duplication!
</span>} </span>else </span>{
</span>    println!("</span>queue is in an unknown state</span>");
</span>}
</span></code></pre>
It's the same functionality as before, but provided as a natural extension to
&&</code>-based if..let</code>-chaining. Now once more, let's try rewriting this as a
match</code> expression:</p>
match</span> state {
</span>    QueueState::Online(count) && !</span>is_full</span>(count) => {
</span>        println!("</span>queue is online and not full</span>");
</span>    }
</span>    QueueState::Online(count) && </span>is_full</span>(count) || </span>// <- logical OR!
</span>    QueueState::Offline(count) && </span>is_full</span>(count) => {
</span>        println!("</span>queue is full</span>");
</span>    }
</span>    _ => println!("</span>queue is in an unknown state</span>"),
</span>}
</span></code></pre>
What's neat here is that because logical  AND (&&</code>) takes precedence over
logical OR (||</code>)
(ref</a>),
we don't need to resort to wrapping our patterns in order to chain them. Oh and
I guess of course it's also nice that just about every programmer regardless of
familiarity with Rust should be able to parse what is going on here.</p>
But even more than that: look at this! This is code that I want to be writing.
It's not trying to be clever or inventing anything particular new. It's defining
feature is just how ordinary this looks. And that really agrees with my
sensibilities for what a general-purpose programming language should feel like.</p>
if let</code>-evaluation ordering</h2>
I don't like reading if-let</code> statements because they flip the read order from
right-to-left. I double don't like reading if-let</code> statements when chained
because they need to be read center-left-right. I like the is</code>-operator because
it allows us to fix that. Let me show you how by taking our last if let..else</code>
example, with all comments stripped:</p>
if let </span>QueueState::Online(count) = state && !</span>is_full</span>(count) {
</span>    println!("</span>queue is online and not full</span>");
</span>} </span>else if let </span>QueueState::Online(count) = state && </span>is_full</span>(count) || 
</span>    </span>let </span>QueueState::Offline(count) = state && </span>is_full</span>(count)
</span>{
</span>    println!("</span>queue is full</span>");
</span>} </span>else </span>{
</span>    println!("</span>queue is in an unknown state</span>");
</span>}
</span></code></pre>
However much I like the boolean operators here, I don't like the order in which
statements evaluate. The first line in our example above is evaluated as follows:</p>
if let </span>QueueState::Online(cond) = state && !</span>is_full</span>(count) { .. }
</span>//     ^^^^^^^^^^^^^^^^^^^^^^^^   ^^^^^    ^^^^^^^^^^^^^^^
</span>//                |                 |              |
</span>//                |       this is evaluated first  |
</span>//                |                                |
</span>//     this is evaluated second                    |
</span>//                                                 |
</span>//                                        this is evaluated last
</span></code></pre>
RFC 3573</a>
proposes to add the is</code>-operator to the language, inspired by C#'s feature of
the same name. This would fix the evaluation order, making the same code instead
read from top-to-bottom, right-to-left:</p>
if</span> state is QueueState::Online(count) && !</span>is_full</span>(count) {
</span>    println!("</span>queue is online and not full</span>");
</span>} </span>else if</span> state is QueueState::Online(count) && </span>is_full</span>(count) || 
</span>    state is QueueState::Offline(count) && </span>is_full</span>(count)
</span>{
</span>    println!("</span>queue is full</span>");
</span>} </span>else </span>{
</span>    println!("</span>queue is in an unknown state</span>");
</span>}
</span></code></pre>
To me this is Rust as it should be. The point has always been to make a language
that is as practical as possible to use, without compromising on correctness,
performance, and control. Making evaluation orders easier to follow seems like
it's right in line with that goal. Here is that same line annotated, but using the is</code> notation:</p>
if</span> state is QueueState::Online(cond) && !</span>is_full</span>(count) { .. }
</span>// ^^^^^    ^^^^^^^^^^^^^^^^^^^^^^^^    ^^^^^^^^^^^^^^^
</span>//   |                         |                |
</span>//  this is evaluated first    |                |        
</span>//                             |                |
</span>//                  this is evaluated second    |
</span>//                                              |
</span>//                                    this is evaluated last
</span></code></pre>
It also comes with an added benefit: it makes it so refactoring code from
if..else</code> form to match</code> form produces an incredibly small diff. It's mostly
the same code, but with the start of each condition trimmed:</p>
match</span> state {
</span>    QueueState::Online(count) && !</span>is_full</span>(count) => {
</span>        println!("</span>queue is online and not full</span>");
</span>    }
</span>    QueueState::Online(count) && </span>is_full</span>(count) ||
</span>    QueueState::Offline(count) && </span>is_full</span>(count) => {
</span>        println!("</span>queue is full</span>");
</span>    }
</span>    _ => println!("</span>queue is in an unknown state</span>"),
</span>}
</span></code></pre>
This would make it possible to explain conditional control flow more
incrementally. I see the potential for carving out a little didactic staircase
into Rust's famously steep learning curve that goes a little something like
this:</p>

Start by introducing if..else</code>, which most programmers will already know.</li>
Then introduce is</code> as a more powerful version of typeof</code> in other languages.</li>
Finally introduce match</code> as a more ergonomic version of if..else</code> + is</code>. Like switch</code> in other languages, but exhaustively checking all cases.</li>
</ol>
We never take any concepts away or have big changes in notation. Each concept
neatly builds on the previous one, and we don't need to wait to introduce if</code>-let until much later like we do now</a>.</p>
Closing Words</h2>
Another thing I wasn't quite sure when to talk about when it comes to the guard
patterns RFC is its interaction with pattern types. Bringing if</code>-guards into
pattern feels like it changes the pattern notation from a fairly
tractable/limited subset that we can evaluate in reasonable time into a system
for arbitrary pre/post-conditions. Unless of course we disallow those kinds of
patterns in pattern types - which creates diverging pattern notations.</p>
Overall I feel like we have a lot to gain in the language by making match</code>-expressions
feel more consistent with the rest of the conditional expressions. After writing
the bulk of this post, I remembered that what I'm sharing here is actually not a
unique insight. RFC
3796</a>
cfg_logical_ops</code> by Jacob Pratt proposes the addition of the boolean &&</code>,
||</code>, and !</code> operators in cfg</code> attributes. Effectively replacing the existing
domain-specific all()</code>, any()</code>, and not()</code> operators.</p>
// current
</span>#[</span>cfg</span>(</span>all</span>(</span>any</span>(a, b), </span>any</span>(c, d), </span>not</span>(e)))]
</span>struct </span>MyStruct {}
</span>
</span>// proposed
</span>#[</span>cfg</span>((a || b) && (</span>c</span> || </span>d</span>) && !</span>e</span>)]
</span>struct </span>MyStruct {}
</span></code></pre>
I like what that RFC proposes. I doubly like it when we consider it as part of a
broader effort to reduce the amount of
micro-syntaxes</a>
spread throughout the language. In turn making Rust an easier language, by
virtue of it being smaller and more consistent.</p>

Syntactic Musings on View Types
Fri, 04 Apr 2025 00:00:00 +0000
Here's a silly little insight I had the other day: if you squint, both View
Types and Pattern Types seem like lightweight forms of Refinement Types 1</a></sup>.
Both will enable constraining types, but in slightly different and complementary
ways. Let's take a look at this using an example of an RGB struct containing
fields for individual Red, Green, and Blue channels stored as usize</code> 2</a></sup>:</p>
^{1</sup>
I like the term "lightweight refinement types" for the category of extensions that include pattern types and view types. To me it's reminiscent of lightweight formal methods</em>: less thorough than the full thing, but with incredible returns for the relative effort expended.</p>
</div>}^{2</sup>
Don't question why we're storing these values as usize</code> too much. It's a little silly. For the purpose of this example just pretend there is some reason why we have to do it like this.</p>
</div>}#[</span>derive</span>(Debug, Default)]
</span>struct </span>Rgb {
</span>    </span>r</span>: </span>usize</span>,
</span>    </span>g</span>: </span>usize</span>,
</span>    </span>b</span>: </span>usize</span>,
</span>}
</span></code></pre>
Pattern types will give us the ability to directly use things like ranges or
enum members in type signatures. In the following example the type usize</code> is
refined</em> to statically only be allowed to contain values between 0..255</code>. This
is similar to the NonZero*</code> types in the stdlib, but as part of the language
and usable with arbitrary patterns:</p>
impl </span>Rgb {
</span>    </span>fn </span>set_red</span>(&</span>mut </span>self</span>, </span>num</span>: </span>usize</span> is ..255) { .. }
</span>    </span>//                                   ^^^^^^^^ range pattern
</span>}
</span></code></pre>
View types are about segmenting the fields contained in self</code> so that multiple
(mutable) methods can operate on the same type without resulting in borrow
checking issues. In the following example we provide mutable access to
individual fields using separate methods. None of these methods take overlapping
borrows of the fields in Self</code>. This means we're free to call all of these
methods, observe their return types, and we won't get any borrow checker errors.
Here's an example using the syntax from Niko's latest
post</a>:</p>
impl </span>Rgb {
</span>    </span>fn </span>red_mut</span>(</span>self</span>: &</span>mut</span> { r } </span>Self</span>) -> .. { .. }
</span>    </span>//                    ^^^^^ view
</span>    </span>fn </span>green_mut</span>(</span>self</span>: &</span>mut</span> { g } </span>Self</span>) -> .. { .. }
</span>    </span>//                      ^^^^^ view
</span>    </span>fn </span>blue_mut</span>(</span>self</span>: &</span>mut</span> { b } </span>Self</span>) -> .. { .. }
</span>    </span>//                     ^^^^^ view
</span>}
</span></code></pre>
Here's a fun question: what happens if we combine combine Pattern Types and View
Types? Both serve different purposes, and I know I've got cases where I'd like
to combine them. So what would that look like? In the abstract, it seems like we
would end up with something like the following:</p>
impl </span>Rgb {
</span>    </span>fn </span>foo</span>(</span>self</span>: &</span>mut</span> { r, g } </span>Self</span> is </span>Self</span> { </span>r</span>: ..255, </span>g</span>: ..255, .. }) {}
</span>    </span>//                ^^^^^^^^      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
</span>    </span>//                  view                      pattern
</span>}
</span></code></pre>
To me that seems like a lot to read. But also rather... double? Both View Types
and Pattern Types refine Self</code> here. I would have expected us to be able to
combine both. Now what if View Types and Pattern Types instead shared the same
syntactic position. There's a reason View Types have to use is</code>, so let's use
that. With that we could rewrite our earlier example like this instead:</p>
impl </span>Rgb {
</span>    </span>fn </span>foo</span>(&</span>mut </span>self</span> is </span>Self</span> { </span>r</span>: ..255, </span>g</span>: ..255, .. }) {}
</span>    </span>//               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
</span>    </span>//                          view + pattern
</span>}
</span></code></pre>
This seems a lot more readable to me. Combining both features seems much less like a hassle here, and maybe even… nice?
This updated notation would also affect our earlier View Types example. Using
the updated notation would now be written the same way, but without using any
patterns:</p>
impl </span>Rgb {
</span>    </span>fn </span>red_mut</span>(&</span>mut </span>self</span> is </span>Self</span> { r, .. }) -> .. { .. }
</span>    </span>//                           ^^^^^^^^^ view
</span>    </span>fn </span>green_mut</span>(&</span>mut </span>self</span> is </span>Self</span> { g, .. }) -> .. { .. }
</span>    </span>//                             ^^^^^^^^^ view
</span>    </span>fn </span>blue_mut</span>(&</span>mut </span>self</span> is </span>Self</span> { b, .. }) -> .. { .. }
</span>    </span>//                            ^^^^^^^^^ view
</span>}
</span></code></pre>
I feel like this would rather neatly unify where we talk about refinements</em> in
function signatures. Notationally this reframes View Types as Pattern Types with ignored fields. Though we don't necessarily need to adopt this notation: what I care about most is that we
think about how both features are intended to come together in the language to
create a cohesive experience. Thanks for reading!</p>

A survey of every iterator variant
Wed, 12 Feb 2025 00:00:00 +0000
Introduction</h2>

Oh yeah, you like iterators? Name all of them. —</p>
</blockquote>
I'm increasingly of the belief that before we can meaningfully discuss the
solution space, we have to first make an effort to build consensus on the
problem space. It's hard to plan for a journey together if we don't know what
the trip will be like along the way. Or worse: if we don't agree in advance
where we want the journey to take us.</p>
In Rust the Iterator</code> trait is the single-most complex trait in the stdlib. It
provides 76 methods, and by my estimate (I stopped counting at 120) has around
150 trait implementations in the stdlib alone. It also features a broad range of
extension traits like FusedIterator</code> and ExactSizeIterator</code> that provide
additional capabilities. And it is itself a trait-based expression of one of
Rust's core control-flow effects; meaning it is at the heart of a lot of
interesting questions about how we combine such effects.</p>
Despite all that we know the Iterator</code> trait falls short today and we would
like to extend it to do more</em>. Async iteration is one popular capability people
are missing as a built-in. Another is the ability to author address-sensitive
(self-referential) iterators. Less in the zeitgeist but equally important are
iterators that can lend items with a lifetime, conditionally terminate
early, and take external arguments on each call to next</code>.</p>
I'm writing this post to enumerate all of the iterator variants in use in Rust
today. So that we can scope the problem space to account for all of them. I'm
also doing this because I've started hearing talk about potentially
(soft-)deprecating Iterator</code>. I believe we only get one shot at running such a
deprecation 1</a></sup>, and if we are going to do one we need to make sure it's a
solution that will plausibly get us through all of our known limitations. Not just one or two.</p>
So without further ado: let's enumerate every variation of the existing
Iterator</code> trait we know we want to write, and discuss how those variants
interact with each other to create interesting new kinds of iterators.</p>
^{1</sup>
I'm basing this off of my experience being a part of the Node.js Streams WG through the mid-2010s. By my count Node.js now features five distinct generations of stream abstractions. Unsurprisingly it's not particularly pleasant to use, and ideally something we should avoid replicating in Rust. Though I'm not opposed to running deprecations of core traits in Rust either, I believe that in order to do that we want to be approaching 100% certainty that it'll be the last deprecation we do. If we're going to embark on a decade plus of migration issues, we have to make absolutely sure it's worth it.</p>
</div>}Base Iterator</h2>
Iterator</code> is a trait representing the stateful component of iteration;
IntoIterator</code> represents the capability for a type to be iterated over. Here is
the Iterator</code> trait as found in the stdlib today. The Iterator</code> and
IntoIterator</code> traits are closely linked, so throughout this post we will show
both to paint a more complete picture.</p>
Throughout this post we will be covering variations on Iterator</code> which provide
new capabilities. This trait is represents the absence of those capabilities: it
is blocking, cannot be evaluated during compilation, is strictly sequential, and
so on. Here is what the foundation of the core::iter</code> submodule looks like today:</p>
/// An iterable type.
</span>pub trait </span>IntoIterator {
</span>    </span>/// The type of elements yielded from the iterator.
</span>    </span>type </span>Item;
</span>    </span>/// Which kind of iterator are we turning this into?
</span>    </span>type </span>IntoIter: Iterator<Item = </span>Self::</span>Item>;
</span>    </span>/// Returns an iterator over the elements in this type.
</span>    </span>fn </span>into_iter</span>(</span>self</span>) -> </span>Self::</span>IntoIter;
</span>}
</span>
</span>/// A type that yields values, one at the time.
</span>pub trait </span>Iterator {
</span>    </span>/// The type of elements yielded from the iterator.
</span>    </span>type </span>Item;
</span>    </span>/// Advances the iterator and yields the next value.
</span>    </span>fn </span>next</span>(&</span>mut </span>self</span>) -> Option<</span>Self::</span>Item>;
</span>    </span>/// Returns the bounds on the remaining length of the iterator.
</span>    </span>fn </span>size_hint</span>(&</span>self</span>) -> (</span>usize</span>, Option<</span>usize</span>>) { .. }
</span>}
</span></code></pre>
While not strictly necessary: the associated IntoIterator::Item</code> type exists
for convenience. That way people using the trait can directly specify the Item</code>
using impl IntoIterator<Item = Foo></code>, which is a lot more convenient than I: <IntoIterator::IntoIterator as Iterator>::Item</code>, etc.</p>
Bounded Iterator</h2>
The base Iterator</code> trait represents a potentially infinite (unbounded) sequence
of items. It has a size_hint</code> method that returns how many items the iterator
still expects to yield. That value is however not guaranteed to be correct, and is
intended to be used for optimizations only. From the docs:</p>

Implementation notes</strong>
It is not enforced that an iterator implementation yields the declared number of elements. A buggy iterator may yield less than the lower bound or more than the upper bound of elements.</p>
size_hint()</code> is primarily intended to be used for optimizations such as reserving space for the elements of the iterator, but must not be trusted to e.g., omit bounds checks in unsafe code. An incorrect implementation of size_hint()</code> should not lead to memory safety violations.</p>
</blockquote>
The Rust stdlib provides two Iterator</code> subtraits that allow it to guarantee it
is bounded: ExactSizeIterator</code> (stable) and TrustedLen</code> (unstable). Both
traits take : Iterator</code> as its super-trait bound and on its surface seem nearly
identical. But there is one key difference: TrustedLen</code> is unsafe to implement allowing it to be used to guarantee safety invariants.</p>
/// An iterator that knows its exact length.
</span>pub trait </span>ExactSizeIterator: Iterator {
</span>    </span>/// Returns the exact remaining length of the iterator.
</span>    </span>fn </span>len</span>(&</span>self</span>) -> </span>usize </span>{ .. }
</span>    </span>/// Returns `true` if the iterator is empty.
</span>    </span>fn </span>is_empty</span>(&</span>self</span>) -> </span>bool </span>{ .. }
</span>}
</span>
</span>/// An iterator that reports an accurate length using `size_hint`.
</span>#[</span>unstable</span>(feature = "</span>trusted_len</span>")]
</span>pub unsafe trait </span>TrustedLen: Iterator {}
</span></code></pre>
ExactSizeIterator</code> has the same memory-safety guarantees as
Iterator::size_hint</code> meaning: you can't rely on it to be correct. That means
that if you're say, collecting items from an iterator into a vec, you can't omit bounds
checks and use ExactSizeIterator::len</code> as the input to
Vec::set_len</code></a>.
However if TrustedLen</code> is implemented bounds checks can be omitted, since the
value returned by size_hint</code> is now a safety invariant of the iterator.</p>
Fused Iterator</h2>
When working with an iterator, we typically iterate over items until the
iterator yields None</code> at which point we treat the iterator as "done". However the documentation for Iterator::next</code> includes the following:</p>

Returns None</code> when iteration is finished. Individual iterator implementations may choose to resume iteration, and so calling next()</code> again may or may not eventually start returning Some(Item)</code> again at some point.</p>
</blockquote>
It's rare to work with iterators which yield None</code> and then resume again
afterwards, but the iterator trait explicitly allows it. Just like it allows for
an iterator to panic if next</code> is called again after None</code> has been yielded
once. Luckily the FusedIterator</code> subtrait exists, which guarantees that once
None</code> has been yielded once from an iterator, all future calls to next</code> will
continue to yield None</code>.</p>
/// An iterator that always continues to yield `None` when exhausted.
</span>pub trait </span>FusedIterator: Iterator {}
</span></code></pre>
Most iterators in the stdlib implement FusedIterator</code>. For iterators which
aren't fused, it's possible to call the Iterator::fuse</code> combinator. The stdlib
docs recommend never taking a FusedIterator</code> bound, instead favoring Iterator</code>
in bounds and calling fuse</code> to guarantee the fused behavior. For iterators that
already implement FusedIterator</code> this is considered a no-op.</p>
The FusedIterator</code> design works because
Option::None</code> is idempotent</em>: there is never a scenario where we can't create a
new instance of None</code>. Contrast this with enums such as
Poll</code></a> that lack a "done"
state -  and you see ecosystem traits such as
FusedFuture</code></a>
attempting to add this lack of expressivity back through other means. The need
for an idempotent "done" state will be important to keep in mind as we explore
other iterator variants throughout this post.</p>
Thread-Safe Iterator</h2>
Where bounded and fused iterators can be obtained by refining the Iterator</code>
trait using subtraits, thread-safe iterators are obtained by composing the
Send</code> and Sync</code> auto-traits with the Iterator</code> trait. That means there is no
need for dedicated SendIterator</code> or SyncIterator</code> traits. A "thread-safe
iterator" becomes a composition of Iterator</code> and Send</code> / Sync</code>:</p>
struct </span>Cat;
</span>
</span>// Manual `Iterator` impl
</span>impl </span>Iterator </span>for </span>Cat { .. }
</span>
</span>// These impls are implied by `Send`
</span>// and `Sync` being auto-traits
</span>unsafe impl </span>Send </span>for </span>Cat {}
</span>unsafe impl </span>Sync </span>for </span>Cat {}
</span></code></pre>
And when taking impls in bounds, we can again express our intent by composing
traits in bounds. I've made the argument before that taking Iterator</code> directly
in bounds is rarely what people actually want, so the bound would end up looking
like this:</p>
fn </span>thread_safe_sink</span>(</span>iter</span>: impl IntoIterator + Send) { .. }
</span></code></pre>
If we also want the individual items yielded by the iterator to be marked
thread-safe, we have to add additional bounds:</p>
fn </span>thread_safe_sink</span><I>(</span>iter</span>: I)
</span>where
</span>    I: IntoIterator<Item: Send> + Send,
</span>{ .. }
</span></code></pre>
While there is no lack of expressivity here, at times bounds like these can get
rather verbose. That should not be taken as indictment of the system itself, but
rather as a challenge to improve the ergonomics for the most common cases.</p>
Dyn-Compatible Iterator</h2>
Dyn-compatibility is another axis that traits are divided on. Unlike e.g.
thread-safety, dyn-compatibility is an inherent part of the trait and is
governed by Sized</code> bounds. Luckily  both the Iterator</code> and IntoIterator</code>
traits are inherently dyn-compatible. That means they can be used to create
trait objects using the dyn</code> keyword:</p>
struct </span>Cat;
</span>impl </span>Iterator </span>for </span>Cat { .. }
</span>
</span>let</span> cat = Cat {};
</span>let</span> dyn_cat: &dyn Iterator = &cat; </span>// ok
</span></code></pre>
There are some iterator combinators such as
count</code></a>
that take an additional Self: Sized</code> bound 2</a></sup>. But because trait objects are
themselves sized, it all mostly works as expected:</p>
^{2</sup>
Admittedly my knowledge of dyn</code> is spotty at best. Out of everything in this post, the section on dyn</code> was the bit I had the least intuition about.</p>
</div>}let mut</span> cat = Cat {};
</span>let</span> dyn_cat: &</span>mut</span> dyn Iterator = &</span>mut</span> cat;
</span>assert_eq!(dyn_cat.</span>count</span>(), </span>1</span>); </span>// ok
</span></code></pre>
Double-Ended Iterator</h2>
Often times iterators over collections hold all data in memory, and can be
traversed in either direction. For this purpose Rust provides the
DoubleEndedIterator</code> trait. Where Iterator</code> builds on the next</code> method,
DoubleEndedIterator</code> builds on a next_back</code> method. This allows items to be
taken from both the logical start and end of the iterator. And once both cursors
meet, iteration is considered over.</p>
/// An iterator able to yield elements from both ends.
</span>pub trait </span>DoubleEndedIterator: Iterator {
</span>    </span>/// Removes and  an element from the end of the iterator.
</span>    </span>fn </span>next_back</span>(&</span>mut </span>self</span>) -> Option<</span>Self::</span>Item>;
</span>}
</span></code></pre>
While you'd expect this trait to be implemented for e.g. VecDeque</code>, it's
interesting to note that it's also implemented for Vec</code>, String</code>, and other
collections that only grow in one direction. Also unlike some of the other
iterator refinements we've seen, DoubleEndedIterator</code> has a required method
that is used as the basis for several new methods such as
rfold</code></a>
(reverse fold</code>) and
rfind</code></a>
(reverse find</code>).</p>
Seeking Iterator</h2>
Both the Iterator</code> and Read</code>traits in Rust provide abstractions for streaming
iteration. The main difference is that Iterator</code> works by returning arbitrary
owned types when calling next</code>, where Read</code> is limited to reading bytes into
buffers. But both abstractions keep a cursor that keeps track of which data has
been processed already, and which data still needs processing.</p>
But not all streams are created equally. When we read data from a regular file 3</a></sup>
on disk we can be sure that, as long as no writes occurred, we can read the same
file again and get the same output. That same guarantee is however not true for
sockets, where once we've read data from it we can't read that same data again.
In Rust this distinction is surfaced via the Seek</code> trait, which gives control
over the Read</code> cursor in types that support it.</p>
^{3</sup>
No, files in /proc</code> are not "regular files". Yes, I know sockets are technically file descriptors.</p>
</div>}In Rust the Iterator</code> trait provides no mechanism to control the underlying
cursor, despite its similarities to Read</code>. A language that does provide an
abstraction for this is C++ in the form of
random_access_iterator</code></a>.
This is a C++ concept (trait-alike) that further refines
bidirectional_iterator</code>. My C++ knowledge is limited, so I'd rather quote the
docs directly than attempt to paraphrase:</p>

[...] random_access_iterator refines bidirectional_iterator by adding support for constant time advancement with the +=</code>, +</code>, -=</code>, and -</code> operators, constant time computation of distance with -</code>, and array notation with subscripting []</code>.</p>
</blockquote>
Being able to directly control the cursor in Iterator</code> implementations could
prove useful when working with in-memory collection types like Vec</code>, as well as
when working with remote objects such as paginated API endpoints. The obvious
starting point for such a trait would be to mirror the existing io::Seek</code> trait
and adapt it to be a subtrait of Iterator</code>:</p>
/// Enumeration of possible methods to seek within an iterator.
</span>pub enum </span>SeekFrom {
</span>    </span>/// Sets the offset to the specified index.
</span>    Start(</span>usize</span>),
</span>    </span>/// Sets the offset to the size of this object plus the specified index.
</span>    End(</span>isize</span>),
</span>    </span>/// Sets the offset to the current position plus the specified index.
</span>    Current(</span>isize</span>),
</span>}
</span>
</span>/// An iterator with a cursor which can be moved.
</span>pub trait </span>SeekingIterator: Iterator {
</span>    </span>/// Seek to an offset in an iterator.
</span>    </span>fn </span>seek</span>(&</span>mut </span>self</span>, </span>pos</span>: SeekFrom) -> Result<</span>usize</span>>;
</span>}
</span></code></pre>
Compile-Time Iterator</h2>
In Rust we can use const {}</code> blocks to execute code during compilation. Only
const fn</code> functions can be called from const {}</code> blocks. const fn</code> free
functions and methods are stable, but const fn</code> trait methods are not. This
means that traits like Iterator</code> are not yet callable from const {}</code> blocks,
and so neither are for..in</code> expressions.</p>
We know we want to support iteration in const {}</code> blocks, but we don't yet know
how we want to spell both the trait declaration and trait bounds. The most
interesting open question here is how we will end up passing the Destruct</code>
trait</a> around, which
is necessary to enable types to be droppable in const</code> contexts. This leads to
additional questions around whether const trait bounds should imply const
Destruct</code>. And whether the const</code> annotation should be part of the trait
declaration, the individual methods, or perhaps both.</p>
This post is not the right venue to discuss all tradeoffs. But to give a sense
of what a compile-time-compatible Iterator</code> trait might look like: here is a
variant where both the trait and individual methods are annotated with const</code>:</p>
pub const trait </span>IntoIterator {                        </span>// ← `const`
</span>    </span>type </span>Item;
</span>    </span>type </span>IntoIter: </span>const </span>Iterator<Item = </span>Self::</span>Item>; </span>// ← `const`
</span>    </span>const fn </span>into_iter</span>(</span>self</span>) -> </span>Self::</span>IntoIter;       </span>// ← `const`
</span>}
</span>
</span>pub const trait </span>Iterator {                                     </span>// ← `const`
</span>    </span>type </span>Item;
</span>    </span>const fn </span>next</span>(&</span>mut </span>self</span>) -> Option<</span>Self::</span>Item>;            </span>// ← `const`
</span>    </span>const fn </span>size_hint</span>(&</span>self</span>) -> (</span>usize</span>, Option<</span>usize</span>>) { .. } </span>// ← `const`
</span>}
</span></code></pre>
Lending Iterator</h2>
While there are plenty of iterators in the stdlib today that yield references to
items, the values that are being referenced are never owned by the iterator
itself. In order to write iterators which owns the items it yields and yields
them by-reference, the associated item type in iterator needs a lifetime:</p>
pub trait </span>IntoIterator {
</span>    </span>type </span>Item<</span>'a</span>>                                               </span>// ← Lifetime
</span>    </span>where
</span>        </span>Self</span>: </span>'a</span>;                                               </span>// ← Bound
</span>    </span>type </span>IntoIter: </span>for</span><</span>'a</span>> Iterator<Item<</span>'a</span>> = </span>Self::</span>Item<</span>'a</span>>>; </span>// ← Lifetime
</span>    </span>fn </span>into_iter</span>(</span>self</span>) -> </span>Self::</span>IntoIter;
</span>}
</span>
</span>pub trait </span>Iterator {
</span>    </span>type </span>Item<</span>'a</span>>                                 </span>// ← Lifetime
</span>    </span>where
</span>        </span>Self</span>: </span>'a</span>;                                 </span>// ← Bound
</span>    </span>fn </span>next</span>(&</span>mut </span>self</span>) -> Option<</span>Self::</span>Item<'_>>; </span>// ← Lifetime
</span>    </span>fn </span>size_hint</span>(&</span>self</span>) -> (</span>usize</span>, Option<</span>usize</span>>) { .. }
</span>}
</span></code></pre>
There has been talk about adding a 'self</code> lifetime to the language as a
shorthand for where Self:'a</code>. But even with that addition this set of
signatures is not for the faint of heart. The reason for that is that it makes
use of GATs, which are both useful and powerful — but are for all intents and
purposes an expert-level type-system feature, and can be a little tricky to
reason about.</p>
Lending iteration is also going to be important when we add dedicated syntax to
construct iterators using the yield</code> keyword. The following example shows a
gen {}</code> block which creates a string and then yields a reference to it. This
perfectly4</a></sup> matches onto the Iterator</code> trait we’ve defined:</p>
^{4</sup>
Remember that strings are heap-allocated so we don’t need to account for address-sensitivity. While the compiler can't yet perform this desugaring, there is nothing in the language prohibiting it.</p>
</div>}let</span> iter = gen {
</span>    </span>let</span> name = String::from("</span>Chashu</span>");
</span>    </span>yield</span> &name; </span>// ← Borrows a local value stored on the heap.
</span>};
</span></code></pre>
Iterator with a Return Value</h2>
The current Iterator</code> trait has an associated type Item</code>, which maps to the
yield</code> keyword in Rust. But it has no associated type that maps to the return</code>
keyword. A way to think about iterators today is that their return type is
hard-coded to unit. If we want to enable generator functions and blocks to be
written which can not just yield</code>, but also return</code>, we'll need some way to
express that.</p>
let</span> counting_iter = gen {
</span>    </span>let mut</span> total = </span>0</span>;
</span>    </span>for</span> item in iter {
</span>        total += </span>1</span>;
</span>        </span>yield</span> item;      </span>// ← Yields one type.
</span>    }
</span>    total                </span>// ← Returns another type.
</span>};
</span></code></pre>
The obvious way to write a trait for "an iterator which can return" is to give
Iterator</code> an additional associated item Output</code> which maps to the logical
return value. In order to be able to express fuse semantics, the function next</code>
needs to be able to return three different states:</p>

Yielding the associated type Item</code></li>
Returning the associated type Output</code></li>
The iterator has been exhausted (done)</li>
</ol>
One way to do this would be for next</code> to return Option<ControlFlow></code>, where
Some(Continue)</code> maps to yield</code>, Some(Break)</code> maps to return</code>, and None</code>
maps to done. Without a final "done" state, calling next</code> again on an iterator
after it's finished yielding values would likely always have to panic. This is
what most futures do in async Rust, and is going to be a problem if we ever want
to guarantee panic-freedom.</p>
pub trait </span>IntoIterator {
</span>    </span>type </span>Output;                                            </span>// ← Output
</span>    </span>type </span>Item;
</span>    </span>type </span>IntoIter:
</span>        Iterator<Output = </span>Self::</span>Output, Item = </span>Self::</span>Item>; </span>// ← Output
</span>    </span>fn </span>into_iter</span>(</span>self</span>) -> </span>Self::</span>IntoIter;
</span>}
</span>
</span>pub trait </span>Iterator {
</span>    </span>type </span>Output;                                           </span>// ← Output
</span>    </span>type </span>Item;
</span>    </span>fn </span>next</span>(&</span>mut </span>self</span>)
</span>        -> Option<ControlFlow<</span>Self::</span>Output, </span>Self::</span>Item>>;  </span>// ← Output
</span>    </span>fn </span>size_hint</span>(&</span>self</span>) -> (</span>usize</span>, Option<</span>usize</span>>) { .. }
</span>}
</span></code></pre>
The way this would work on the caller side and in combinators is quite
interesting. Starting with the provided
for_each</code></a>
method: it will want to return Iterator::Output</code> rather than ()</code> once the
iterator has completed. Crucially the closure F</code> provided to for_each</code> only
operates on Self::Output</code>, and has no knowledge of Self::Output</code>. Because if
the closure did have direct knowledge of Output</code>, it could short-circuit by
returning Output</code> sooner than expected which is a different kind of iterator
than having a return value.</p>
fn </span>for_each</span><F>(</span>self</span>, </span>f</span>: F) -> </span>Self::</span>Output </span>// ← Output
</span>where
</span>    </span>Self</span>: Sized,
</span>    F: FnMut(</span>Self::</span>Item),
</span>{ .. }
</span></code></pre>
If we were to transpose "iterator with return value" to for..in</code> things get
even more interesting still. In Rust loop</code> expressions can themselves evaluate
to non-unit types by calling break</code> with some value. for..in</code> expressions
cannot do this in the general case yet, except for the handling of errors using
?</code>. But it's not hard to see how this could be made to work, conceptually this
is equivalent to calling Iterator::try_for_each</code></a> and returning
Some(Break(value))</code>:</p>
let</span> ty: </span>u32 </span>= </span>for</span> item in iter {  </span>// ← Evaluates to a `u32`
</span>    </span>break </span>12</span>u32                   </span>// ← Calls `break` from the loop body
</span>};
</span></code></pre>
Assuming we have an Iterator</code> with its own return value, that would mean
for..in</code> expressions would be able to evaluate to non-unit return types without
calling break</code> from within the loop's body:</p>
let</span> ty: </span>u32 </span>= </span>for</span> item in iter {  </span>// ← Evaluates to a `u32`
</span>    dbg!(item);                   </span>// ← Doesn't `break` from the loop body
</span>};
</span></code></pre>
This of course leads to questions about how to combine an "iterator with a
return value" and "using break</code> from inside a for..in</code> expression". I'll leave
that as an exercise to the reader on how to spell out (I'm certain it can be
done, I just think it's a fun one). Generalizing all modes of early returns from
for..in</code> expressions to invocations of for_each</code> combinators is an interesting
challenge that we'll cover in more detail later on when we discuss
short-circuiting (fallible) iterators.</p>
Iterator with a Next Argument</h2>
In generator blocks the yield</code> keyword can be used to repeatedly yield values
from an iterator. But what if the caller doesn't just want to obtain values, but
also pass new values back into the iterator? That would require for yield</code> to be able
evaluate to a non-unit type. Iterators that have this functionality are often
referred to as "coroutines", and they are being particularly useful when implementing I/O-Free Protocols</a>.</p>
/// Some RPC protocol
</span>enum </span>Proto {
</span>    </span>/// Some protocol state
</span>    MsgLen(</span>u32</span>),
</span>}
</span>
</span>let</span> rpc_handler = gen {
</span>    </span>let</span> len = message.</span>len</span>();
</span>    </span>let</span> next_state = </span>yield</span> Proto::MsgLen(len); </span>// ← `yield` evaluates to a value
</span>    ..
</span>};
</span></code></pre>
In order to support this, Iterator::next</code> needs to be able to take an
additional argument in the form of a new associated type Args</code>. This associated
type has the same name as the input arguments to Fn</code> and AsyncFn</code>. If
"iterator with a next argument" can be thought of as representing a "coroutine",
the Fn</code> family of traits can be thought of as representing a regular "routine"
(function).</p>
pub trait </span>IntoIterator {
</span>    </span>type </span>Item;
</span>    </span>type </span>Args;                                          </span>// ← Args
</span>    </span>type </span>IntoIter:
</span>        Iterator<Item = </span>Self::</span>Item, Args = </span>Self::</span>Args>; </span>// ← Args
</span>    </span>fn </span>into_iter</span>(</span>self</span>) -> </span>Self::</span>IntoIter;
</span>}
</span>
</span>pub trait </span>Iterator {
</span>    </span>type </span>Item;
</span>    </span>type </span>Args;                                                  </span>// ← Args
</span>    </span>fn </span>next</span>(&</span>mut </span>self</span>, </span>args</span>: </span>Self::</span>Args) -> Option<</span>Self::</span>Item>; </span>// ← Args
</span>    </span>fn </span>size_hint</span>(&</span>self</span>) -> (</span>usize</span>, Option<</span>usize</span>>) { .. }
</span>}
</span></code></pre>
Here too it's interesting to consider the caller side. Starting with a
hypothetical for_each</code> function: it would need to be able to take an initial
value passed to next</code>, and then from the closure be able to return additional
values passed to subsequent invocations of next</code>. The signature for that would
look like this:</p>
fn </span>for_each</span><F>(</span>self</span>, </span>args</span>: </span>Self::</span>Args, </span>f</span>: F) </span>// ← Args
</span>where
</span>    </span>Self</span>: Sized,
</span>    F: FnMut(</span>Self::</span>Item) -> </span>Self::</span>Args,      </span>// ← Args
</span>{ .. }
</span></code></pre>
To better illustrate how this would work and build up some intuition, consider
manual calls to next</code> inside of a loop. We would start by constructing some
initial state that is passed to the first invocation of next</code>. This will
produce an item that can be used to construct the next argument to next</code>. And so on, until the iterator has no more items left to yield:</p>
let mut</span> iter = some_value.</span>into_iter</span>();
</span>let mut</span> arg = some_initial_value;
</span>
</span>// loop as long as there are items to yield
</span>while let </span>Some(item) = iter.</span>next</span>(arg) {
</span>    </span>// use `item` and compute the next `arg`
</span>    arg = </span>process_item</span>(item);
</span>}
</span></code></pre>
If we were to iterate over an iterator with a next function using a for..in</code>
expression, the equivalent of returning a value from a closure would be either
to continue</code> with a value. This could potentially also be the final expression
in the loop body, which you can think of itself being an implied continue</code>
today. The only question remaining is how to pass initial values on loop
construction, but that mainly seems like an exercise in syntax design:</p>
// Passing a value to the `next` function seems like
</span>// it would logically map to `continue`-expressions.
</span>// (passing initial state to the loop intentionally omitted)
</span>for</span> item in iter {
</span>    </span>continue </span>process_item</span>(item);   </span>// ← `continue` with value
</span>};
</span>
</span>// You can think of `for..in` expressions as having
</span>// an implied `continue ()` at the end. Like functions
</span>// have an implied `return ()`. What if that could take
</span>// a value?
</span>// (passing initial state to the loop intentionally omitted)
</span>for</span> item in iter {
</span>    </span>process_item</span>(item)             </span>// ← `continue` with value
</span>};
</span></code></pre>
"iterator with return value" and "iterator with next argument" feel like they
map particularly well to break</code> and continue</code>. I think of them as duals,
enabling both expressions to carry non-unit types. This feels like it might be
an important insight that I haven't seen anyone bring up before.</p>
Being able to pass a value to next</code> is one of the hallmark features of the
Coroutine</code> trait in the stdlib. However unlike the trait sketch we provided
here, in Coroutine</code> the type of the value passed to next</code> is defined as a
generic param on the trait rather than an associated type. Presumably that is so
that Coroutine</code> can have multiple implementations on the same type that depend
on the input type. I searched whether this was actually being used today, and
it doesn't seem to
be</a>.
Which is why I suspect it is likely fine to use an associated type for the input
arguments.</p>
Short-Circuiting Iterator</h2>
While it is possible to return Try</code>-types such as Result</code> and Option</code> from an
iterator today, it isn't yet possible is to immediately stop execution in the
case of an error [^not-panic]. This behavior is typically referred to as
"short-circuiting": halting normal operation and triggering an
exceptional state (like an electrical breaker would in a building).</p>
^{5</sup>
By that I mean: an error, not a panic.</p>
</div>
In unstable Rust we have try {}</code> blocks that rely on the unstable Try</code> trait,
but we don't yet have try fn</code> functions. If we want these to function in traits
the way we'd want them to, they will have to desugar to impl Try</code>. Rather than
speculate about what a potential try fn</code> syntax might look like in the future,
we'll be writing our samples using -> impl Try</code> directly. Here is what the
Try</code> (and FromResidual</code>) traits look like today:</p>}/// The `?` operator and `try {}` blocks.
</span>pub trait </span>Try: FromResidual {
</span>    </span>/// The type of the value produced by `?`
</span>    </span>/// when _not_ short-circuiting.
</span>    </span>type </span>Output;
</span>    
</span>    </span>/// The type of the value passed to `FromResidual::from_residual`
</span>    </span>/// as part of ? when short-circuiting.
</span>    </span>type </span>Residual;
</span>    
</span>    </span>/// Constructs the type from its `Output` type.
</span>    </span>fn </span>from_output</span>(</span>output</span>: </span>Self::</span>Output) -> </span>Self</span>;
</span>
</span>    </span>/// Used in ? to decide whether the operator should produce
</span>    </span>/// a value ( or propagate a value back to the caller.    
</span>    </span>fn </span>branch</span>(</span>self</span>) -> ControlFlow<</span>Self::</span>Residual, </span>Self::</span>Output>;
</span>}
</span>
</span>/// Used to specify which residuals can be converted
</span>/// into which `Try` types.
</span>pub trait </span>FromResidual<R = <Self as Try>::Residual> {
</span>    </span>/// Constructs the type from a compatible `Residual` type.
</span>    </span>fn </span>from_residual</span>(</span>residual</span>: R) -> </span>Self</span>;
</span>}
</span></code></pre>
You can think of a short-circuiting iterator as a special case of an "iterator
with return value". In its base form, it will only return early in case of an
exception while its logical return type remains hard-coded to unit. The return
type of fn next</code> should be an impl Try</code> returning an Option</code>, with the value
of Residual</code> set to the associated Residual</code> type. This allows all combinators
to share the same Residual</code>, enabling types to flow.</p>
pub trait </span>IntoIterator {
</span>    </span>type </span>Item;
</span>    </span>type </span>Residual;
</span>    </span>type </span>IntoIter: Iterator<
</span>        Item = </span>Self::</span>Item,
</span>        Residual = Residual   </span>// ← Residual
</span>    >;
</span>    </span>fn </span>into_iter</span>(</span>self</span>) -> </span>Self::</span>IntoIter;
</span>}
</span>
</span>pub trait </span>Iterator {
</span>    </span>type </span>Item;
</span>    </span>type </span>Residual;                   </span>// ← Residual
</span>    </span>fn </span>next</span>(&</span>mut </span>self</span>) -> impl Try<  </span>// ← impl Try
</span>        Output = Option<</span>Self::</span>Item>,
</span>        Residual = </span>Self::</span>Residual,   </span>// ← Residual
</span>    >;
</span>}
</span></code></pre>
If we once again consider the caller, we'll want to provide a way for for..in</code>
expressions to short-circuit. What's interesting here is that the base iterator
trait already provides a
try_for_each</code></a>,
method. The difference between that method and the for_each</code> we're about to see
is how the type of Residual</code> is obtained. In try_for_each</code> the value is local
to the method, while if the trait itself is "short-circuiting" the type of
Residual</code> is determined by the associated Self::Residual</code> type. Or put
differently: in a short-circuiting iterator the type we short-circuit with is a
property of the trait rather than a property of the method.</p>
fn </span>for_each</span><F, R>(</span>self</span>, </span>f</span>: F) -> R                  </span>// ← Return type
</span>where
</span>    </span>Self</span>: Sized,
</span>    F: FnMut(</span>Self::</span>Item) -> R,                      </span>// ← Return type
</span>    R: Try<Output = (), Residual = </span>Self::</span>Residual>, </span>// ← `impl Try`
</span>{ .. }
</span></code></pre>
As mentioned earlier on in this post: the interaction between "iterator with a
return type" and "short-circuiting iterator" is an interesting one. Returning
Option<ControlFlow></code> from fn next</code> is able to encode three distinct states,
but this combination of capabilities requires us to encode four states:</p>

yield a next item</li>
break with a residual</li>
return a final output</li>
iterator done (idempotent)</li>
</ol>
The reason why we want to be able to encode a signature like this is because
when writing gen fn</code> functions it is entirely reasonable to want to have a
return type, short-circuit on error using ?</code>, and also yield</code> values. This
works like regular functions today, but with the added capability to invoke
yield</code>. The naive way of encoding this would be to return an impl Try</code> of
Option<ControlFlow<_>></code> with distinct associated types for Item</code>, Output</code>,
and Residual</code>. This does however feel like it is starting to get a little out
of hand, though perhaps a first-class try fn</code> notation might provide some
relief.</p>
pub trait </span>IntoIterator {
</span>    </span>type </span>Item;
</span>    </span>type </span>Output;                           </span>// ← `Output` 
</span>    </span>type </span>Residual;                         </span>// ← `Residual` 
</span>    </span>type </span>IntoIterator: Iter<
</span>        Item = </span>Self::</span>Item,
</span>        Residual = </span>Self::</span>Residual,         </span>// ← `Residual` 
</span>        Output = </span>Self::</span>Output,             </span>// ← `Output` 
</span>    >;
</span>    </span>fn </span>into_iter</span>(</span>self</span>) -> </span>Self::</span>IntoIter;
</span>}
</span>
</span>pub trait </span>Iterator {
</span>    </span>type </span>Item;
</span>    </span>type </span>Output;                     </span>// ← `Output`
</span>    </span>type </span>Residual;                   </span>// ← `Residual`
</span>    </span>fn </span>next</span>(&</span>mut </span>self</span>) -> impl Try<  </span>// ← `impl Try` 
</span>        Output = Option<ControlFlow< </span>// ← `ControlFlow
</span>            </span>Self::</span>Output,            </span>// ← `Output
</span>            </span>Self::</span>Item,
</span>        >>,
</span>        Residual = </span>Self::</span>Residual,   </span>// ← `Residual` 
</span>    >;
</span>}
</span></code></pre>
Address-Sensitive Iterator</h2>
Rust's generator transformation may create self-referential types. That is:
types which have fields that borrow from other fields on the same type. We call
these types "address-sensitive" because once the type has been constructed, its
address in memory must remain stable. This comes up when writing gen {}</code> blocks
that have stack-allocated locals 6</a></sup> that are kept live across
yield</code>-expressions. What is or isn't a "stack-allocated local" can get a little
complicated. But it's important to highlight that for example calling
IntoIterator::into_iter</code> on a type and re-yielding all items is something that
just works
(playground</a>):</p>
^{6</sup>
This affects heap-allocated locals too, but that's not a limitation of the language, only one of the impl.</p>
</div>}// This example works today
</span>let</span> iter = gen {
</span>    </span>let</span> cat_iter = cats.</span>into_iter</span>();
</span>    </span>for</span> cat in cat_iter {
</span>        </span>yield</span> cat;
</span>    }
</span>};
</span></code></pre>
And to give a sense of what for example does not work, here is one of the
samples Tmandry (T-Lang) collected. This creates an intermediate borrow, which
results in the error: "Borrow may still be in use when gen</code> fn body yields"
(playground</a>):</p>
gen </span>fn </span>iter_set_rc</span><T: Clone>(</span>xs</span>: Rc<RefCell<HashSet<T>>>) ... {
</span>    </span>for</span> x in xs.</span>borrow</span>().</span>iter</span>() {
</span>        </span>yield</span> x.</span>clone</span>();
</span>    }
</span>}
</span></code></pre>
In order to enable examples like the last one to work, Rust needs to be able to
express some form of "address-sensitive iterator". The obvious starting point
would be to mint a new trait PinnedIterator</code> which changes the self-type of
next</code> to take a Pin<&mut Self></code> rather than &mut self</code>:</p>
trait </span>IntoIterator {
</span>    </span>type </span>Item;
</span>    </span>type </span>IntoIter: Iterator<Item = </span>Self::</span>Item>;
</span>    </span>fn </span>into_iter</span>(</span>self</span>) -> </span>Self::</span>IntoIter;
</span>}
</span>
</span>trait </span>Iterator {
</span>    </span>type </span>Item;
</span>    </span>fn </span>next</span>(</span>self</span>: Pin<&</span>mut Self</span>>) -> Option<</span>Self::</span>Item>; </span>// ← `Pin<&mut Self>`
</span>    </span>fn </span>size_hint</span>(&</span>self</span>) -> (</span>usize</span>, Option<</span>usize</span>>) { .. }
</span>}
</span></code></pre>
Enumerating all the problems of Pin</code> is worth its own blog post. But still, it
seems important enough to point out that this definition has what Rust for Linux
calls The Safe Pinned Initialization
Problem</a>.
IntoIterator::into_iter</code> cannot return a type that is address-sensitive at time
of construction, instead address-sensitivity is something that can only be
guaranteed at a later point once the type is pin!</code>ned in place.</p>
At the start of this post I used the phrase: "(soft-)deprecation of the
Iterator</code> trait". With that I was referring to one proposed idea to enable gen {}</code> to return a new trait Generator</code> with the same signature as our example.
As well as some bridging impls for the purposes of compatibility. The core of
the compat system would be as follows:</p>
/// All `Iterator`s are `Generator`s
</span>impl</span><I: IntoIterator> IntoGenerator </span>for </span>I {
</span>    </span>type </span>Item = I::Item;
</span>    </span>type </span>IntoGen = IteratorGenerator<</span>I::</span>IntoIter>;
</span>    </span>fn </span>into_gen</span>(</span>self</span>) -> </span>Self::</span>IntoGen {
</span>        IteratorGenerator(</span>self</span>.</span>into_iter</span>())
</span>    }
</span>}
</span>
</span>/// Only pinned `Generator`s are `Iterator`s
</span>impl</span><G> Iterator </span>for </span>Pin<G>
</span>where
</span>    G: DerefMut,
</span>    </span>G::</span>Target: Generator,
</span>{
</span>    </span>type </span>Item = <<G as Deref>::Target as Generator>::Item;
</span>    </span>fn </span>next</span>(&</span>mut </span>self</span>) -> Option<</span>Self::</span>Item> {
</span>        Generator::next(</span>self</span>.</span>as_mut</span>())
</span>    }
</span>}
</span></code></pre>
This creates a situation that I've been describing as "one-and-a-half-way
compat", as opposed to the usual two-way-compat. And we need two-way-compat to
not be a breaking change. This leads to a situation where changing a bound from
taking Iterator</code> to Generator</code> is backwards-compatible. But changing an impl
from returning an Iterator</code> to returning a Generator</code> is not
backwards-compatible. The obvious solution then would be to migrate the entire
ecosystem to take Generator</code> bounds everywhere. Coupled with gen {}</code> only ever
returning Generator</code> and not Iterator</code>: that is a deprecation of Iterator</code> in
all but name.</p>
At first sight it might seem like we're being forced into deprecating Iterator</code>
because of the limitations of Pin</code>. The obvious answer to that would be to
solve the issues with Pin</code> by replacing it with something
better</a>. But that creates
a false dichotomy: there is nothing forcing us to make a decision on this today.
As we established at the start of this section: a surprising amount of use cases
already work without the need for address-sensitive iterators. And as we've seen
throughout this post: address-sensitive iteration is far from the only feature
that gen {}</code> blocks will not be able to support on day one.</p>
Iterator Guaranteeing Destruct</h2>
The current formulation of thread::scope</code> requires that the thread it’s called
on remains blocked until all threads have been joined. This requires stepping
into a closure and executing all code within that. Contrast this with something
like FutureGroup</code></a>
which logically owns computations and can be freely moved around. The values of
the futures resolved within can in turn be yielded out. But unlike
thread::scope</code> it can’t guarantee that all computations will complete, and so
a parallel version of FutureGroup</code> can't mutably hold onto mutable borrows the
way thread::scope</code> can.</p>
// A usage example of `thread::scope`,
</span>// the ability to spawn new threads
</span>// is only available inside the closure.
</span>thread::scope(|</span>s</span>| {
</span>    s.</span>spawn</span>(|| ..);
</span>    s.</span>spawn</span>(|| ..);
</span>                    </span>// ← All threads are joined here.
</span>});
</span>
</span>// A usage example of `FutureGroup`,
</span>// no closures required.
</span>let mut</span> group = FutureGroup::new();
</span>group.</span>insert</span>(future::ready(</span>2</span>));
</span>group.</span>insert</span>(future::ready(</span>4</span>));
</span>group.</span>for_each</span>(|_| ()).await;
</span></code></pre>
If we want to write a type similar to FutureGroup</code> with the same guarantees as
thread::scope</code>, we'd either need to guarantee that FutureGroup</code> can never be
dropped or guarantee that FutureGroup</code>'s Drop</code> impl is always run. It turns
out that it's rather impractical to have types that can't be dropped in a
language where every function may panic. So the only real option here are to
have types whose destructors are guaranteed to run.</p>
The most plausible way we know of to do this would be by introducing a new auto
trait Leak</code>, disallowing types from being passed to mem::forget</code>, Box::leak</code>,
and so on. For more on the design, read Linear Types
One-Pager</a>. Because Leak</code>
is an auto-trait, we could compose it with the existing Iterator</code> and
IntoIterator</code> traits, similar to Send</code> and Move</code>:</p>
fn </span>linear_sink</span>(</span>iter</span>: impl IntoIterator<IntoIter: </span>?</span>Leak>) { .. }
</span></code></pre>
Async Iterator</h2>
In Rust the async</code> keyword can transform imperative function bodies into state
machines that can be manually advanced by calling the Future::poll</code> method.
Under the hood this is done using what is called a coroutine transform</em>, which
is the same transform we use to desugar gen {}</code> blocks with. But that's
just the mechanics of it; the async</code> keyword in Rust also introduces two new
capabilities</a>: ad-hoc concurrency
and ad-hoc cancellation. Together these capabilities can be combined to create
new control-flow operations, like
Future::race</code></a> and Future::timeout</code>.</p>
Async Functions in Traits were stabilized one year ago in Rust 1.75, enabling
the use of  async fn</code> in traits. Making the Iterator</code>trait work with async</code>
is mostly a matter of adding an async</code> prefix to next</code>:</p>
trait </span>IntoIterator {
</span>    </span>type </span>Item;
</span>    </span>type </span>IntoIter: Iterator<Item = </span>Self::</span>Item>;
</span>    </span>fn </span>into_iter</span>(</span>self</span>) -> </span>Self::</span>IntoIter;
</span>}
</span>
</span>trait </span>Iterator {
</span>    </span>type </span>Item;
</span>    async </span>fn </span>next</span>(&</span>mut </span>self</span>) -> Option<</span>Self::</span>Item>;      </span>// ← async
</span>    </span>fn </span>size_hint</span>(&</span>self</span>) -> (</span>usize</span>, Option<</span>usize</span>>) { .. }
</span>}
</span></code></pre>
While the next</code> method here would be annotated with the async</code> keyword, the
size_hint</code> method should probably not be. The reason for that is that it acts
as a simple getter, and it really shouldn't be performing any asynchronous
computation. It's also unclear whether into_iter</code> should be an async fn</code> or
not. There is probably a pattern to be established here, and it might very well
be.</p>
An combination of iterator variants that's been of some interest recently is an
address-sensitive async iterator. We could imagine writing an address-sensitive
async iterator by making next</code> take self: Pin<&mut Self></code>:</p>
trait </span>IntoIterator {
</span>    </span>type </span>Item;
</span>    </span>type </span>IntoIter: Iterator<Item = </span>Self::</span>Item>;
</span>    async </span>fn </span>into_iter</span>(</span>self</span>) -> </span>Self::</span>IntoIter;
</span>}
</span>
</span>trait </span>Iterator {
</span>    </span>type </span>Item;
</span>    async </span>fn </span>next</span>(</span>self</span>: Pin<&</span>mut Self</span>>) -> Option<</span>Self::</span>Item>; </span>// ← async + `Pin<&mut Self>`
</span>    </span>fn </span>size_hint</span>(&</span>self</span>) -> (</span>usize</span>, Option<</span>usize</span>>) { .. }
</span>}
</span></code></pre>
This signature is likely to confuse some people. async fn next</code> return an impl Future</code> which itself must be pinned prior to being polled. In this example we are
separately requiring that Self</code> is also pinned. That is because "the state of
the iterator" and "the state of the future" are not the same state. We
intuitively understand this when working with non-async address-sensitive
iterators: the locals created within the next</code> are not captured by the
enclosing iterator, and are free to be stack-pinned for the duration of the call
to next</code>. But when working with an asynchronous address-sensitive iterator,
for some reason people seem to assume that all locals defined in fn next</code> now
need to be owned by the iterator and not the future.</p>
In the async Rust ecosystem there exists a popular variation of an async
iterator trait called Stream</code>. Rather than keeping the state of the iterator
(self</code>) and the next</code> function separate, it combines both into a single state.
The trait has a single method poll_next</code> which acts as a mix between
Future::poll</code> and Iterator::next</code>. With a provided convenience function async fn next</code> that is a thin wrapper around poll_next</code>.</p>
trait </span>IntoStream {
</span>    </span>type </span>Item;
</span>    </span>type </span>IntoStream: Stream<Item = </span>Self::</span>Item>;
</span>    </span>fn </span>into_stream</span>(</span>self</span>) -> </span>Self::</span>IntoStream;
</span>}
</span>
</span>pub trait </span>Stream {
</span>    </span>type </span>Item;
</span>    </span>fn </span>poll_next</span>(                            </span>// ← `fn poll_next`
</span>        </span>self</span>: Pin<&</span>mut Self</span>>,                </span>// ← `Pin<&mut Self>`
</span>        </span>cx</span>: &</span>mut </span>Context<'_>,                </span>// ← `task::Context`
</span>    ) -> Poll<Option<</span>Self::</span>Item>>;           </span>// ← `task::Poll` 
</span>    async </span>fn </span>next</span>(&</span>mut </span>self</span>) -> </span>Self::</span>Item   </span>// ← `async`
</span>    </span>where
</span>        </span>Self</span>: Unpin                          </span>// ← `Self: Unpin`
</span>    { .. }
</span>    </span>fn </span>size_hint</span>(&</span>self</span>) -> (</span>usize</span>, Option<</span>usize</span>>) { ... }
</span>}
</span></code></pre>
By combining both states into a single state, this trait violates one of the
core tenets of async Rust's design: the ability to uniformly communicate
cancellation by dropping futures. Here if the future by fn next</code> is dropped,
that is a no-op and cancellation will not occur. This causes compositional async
control-flow operators like Future::race</code> to not work despite compiling.</p>
To instead cancel the current call to next</code> you are forced to either drop the
entire stream, or use some bespoke method to cancel just the future's state.
Cancellation in async Rust is infamous for being hard to get right, which is
understandable when (among other things) core traits in the ecosystem do not
correctly handle it.</p>
Concurrent Iterator</h2>
As we're approaching the end of our exposition here, let's talk about the most
elaborate variations on Iterator</code>. First in line: the rayon</code> crate and the
ParallelIterator</code> trait. Rayon provides what are called "parallel iterators"
which process items concurrently rather than sequentially, using operating
system threads. This tends to significantly improve throughput compared to
sequential processing, but have the caveat that all consumed items must
implement Send</code>. To see just how familiar parallel iterators can be: the
following example looks almost identical to a sequential iterator except for the
call to into_par_iter</code> instead of into_iter</code>.</p>
use </span>rayon::prelude::*;
</span>
</span>(</span>0</span>..</span>100</span>)
</span>    .</span>into_par_iter</span>()   </span>// ← Instead of calling `into_iter`.
</span>    .</span>for_each</span>(|</span>x</span>| println!("</span>{:?}</span>", x));
</span></code></pre>
The ParallelIterator</code> trait however comes as a pair with the Consumer</code> trait.
It can be a little mind-boggling but the way Rayon works is that combinators can
be chained to create a handler, which at the end of the chain is copied to each
thread and used there to handle items. This is of course a simplified
explanation; I'll defer to Rayon maintainers to provide a detailed explanation.
To give you a sense how different these traits are from the regular
Iterator</code>traits, here they are (simplified):</p>
/// A consumer is effectively a generalized “fold” operation.
</span>pub trait </span>Consumer<Item>: Send + Sized {
</span>    </span>/// The type of folder that this consumer can be converted into.
</span>    </span>type </span>Folder: Folder<Item, Result = </span>Self::</span>Result>;
</span>    </span>/// The type of reducer that is produced if this consumer is split.
</span>    </span>type </span>Reducer: Reducer<</span>Self::</span>Result>;
</span>    </span>/// The type of result that this consumer will ultimately produce.
</span>    </span>type </span>Result: Send;
</span>}
</span>
</span>/// A type that can be iterated over in parallel
</span>pub trait </span>IntoParallelIterator {
</span>    </span>/// What kind of iterator are we returning?
</span>    </span>type </span>Iter: ParallelIterator<Item = </span>Self::</span>Item>;
</span>    </span>/// What type of item are we yielding?
</span>    </span>type </span>Item: Send;
</span>        </span>/// Return a stateful parallel iterator.
</span>    </span>fn </span>into_par_iter</span>(</span>self</span>) -> </span>Self::</span>Iter;
</span>}
</span>
</span>/// Parallel version of the standard iterator trait.
</span>pub trait </span>ParallelIterator: Sized + Send {
</span>    </span>/// The type of item that this parallel iterator produces.
</span>    </span>type </span>Item: Send;
</span>    </span>/// Internal method used to define the behavior of this
</span>    </span>/// parallel iterator. You should not need to call this
</span>    </span>/// directly.
</span>    </span>fn </span>drive_unindexed</span><C>(</span>self</span>, </span>consumer</span>: C) -> </span>C::</span>Result
</span>    </span>where
</span>        C: UnindexedConsumer<</span>Self::</span>Item>;
</span>}
</span></code></pre>
What matters most here is that using</em> the ParallelIterator</code> trait feels
similar to a regular iterator. All you need to do is call into_par_iter</code>
instead of into_iter</code> and you're off to the races. On the consuming side it
seems like we should be able to author some variation of for..in</code> to consume
parallel iterators. Rather than speculate about syntax, we can look at the
signature of ParallelIterator::for_each</code> to see which guarantees this would
need to make.</p>
fn </span>for_each</span><F>(</span>self</span>, </span>f</span>: F)
</span>where
</span>    F: Fn(</span>Self::</span>Item) + Sync + Send
</span>{ .. }
</span></code></pre>
We can observe three changes here from the base iterator trait:</p>

Self</code> no longer needs to be Sized</code>.</li>
Somewhat predictably the closure F</code> needs to be thread-safe.</li>
The closure F</code> needs to implement Fn</code> rather than FnMut</code> to prevent data races.</li>
</ol>
We can then infer that in the case of a parallel for..in</code> expression, the loop
body would not be able to close over any mutable references. This is an addition
to the existing limitation that loop bodies already can't express FnOnce</code>
semantics and move values (e.g. "Warning: this value was moved in a previous
iteration of the loop"</em>.)</p>
An interesting combination of are "parallel iteration" and "async iteration".
An interesting aspect of the async</code> keyword in Rust is that it allows for
ad-hoc concurrent execution of futures without needing to rely on special
syscalls or OS threads. This means that concurrency and parallelism can be
detached from one another. While we haven't yet seen a "parallel async iterator"
trait in the ecosystem, the futures-concurrency</code> crate does encode a
"concurrent async iterator" ^{7</a></sup>. Just like ParallelIterator</code>,
ConcurrentAsyncIterator</code> comes in a pair with a Consumer</code>trait.</p>}
^{7</sup>
Technically this trait is called ConcurrentStream</code>, but there is little that is Stream</code>-dependent here. I called it that because it is compatible with the futures_core::Stream</code> trait since futures-concurrency</code> is intended to be a production crate.</p>
</div>}/// Describes a type which can receive data.
</span>pub trait </span>Consumer<Item, Fut>
</span>where
</span>    Fut: Future<Output = Item>,
</span>{
</span>    </span>/// What is the type of the item we’re returning when completed?
</span>    </span>type </span>Output;
</span>    </span>/// Send an item down to the next step in the processing queue.
</span>    async </span>fn </span>send</span>(</span>self</span>: Pin<&</span>mut Self</span>>, </span>fut</span>: Fut) -> ConsumerState;
</span>    </span>/// Make progress on the consumer while doing something else.
</span>    async </span>fn </span>progress</span>(</span>self</span>: Pin<&</span>mut Self</span>>) -> ConsumerState;
</span>    </span>/// We have no more data left to send to the `Consumer`;
</span>    </span>/// wait for its output.
</span>    async </span>fn </span>flush</span>(</span>self</span>: Pin<&</span>mut Self</span>>) -> </span>Self::</span>Output;
</span>}
</span>
</span>pub trait </span>IntoConcurrentAsyncIterator {
</span>    </span>type </span>Item;
</span>    </span>type </span>IntoConcurrentAsyncIter: ConcurrentAsyncIterator<Item = </span>Self::</span>Item>;
</span>    </span>fn </span>into_co_iter</span>(</span>self</span>) -> </span>Self::</span>IntoConcurrentAsyncIter;
</span>}
</span>
</span>pub trait </span>ConcurrentAsyncIterator {
</span>    </span>type </span>Item;
</span>    </span>type </span>Future: Future<Output = </span>Self::</span>Item>;
</span>
</span>    </span>/// Internal method used to define the behavior
</span>    </span>/// of this concurrent iterator. You should not
</span>    </span>/// need to call this directly.
</span>    async </span>fn </span>drive</span><C>(</span>self</span>, </span>consumer</span>: C) -> </span>C::</span>Output
</span>    </span>where
</span>        C: Consumer<</span>Self::</span>Item, </span>Self::</span>Future>;
</span>}
</span></code></pre>
While ParallelIterator</code> and ConcurrentAsyncIterator</code> have similarities in both
usage and design, they are different enough that we cant quite think as one
being the async, non-thread-safe version of the other. Perhaps it is possible to
bring both traits closer to one another, so that the only difference are a few
strategically placed async</code> keywords, but more research is needed to validate
whether that is possible.</p>
Another interesting bit to point out here: concurrent iteration is also mutually
exclusive with lending iteration. A lending iterator relies on yielded items
having sequential lifetimes, while concurrent lifetimes rely on yielded items
having overlapping lifetimes. Those are fundamentally incompatible concepts.</p>
Conclusion</h2>
And that's every iterator variant that I know of, bringing us to 17 different
variations. If we take away the variants that we can express using subtraits (4
variants) and auto-traits (5 variants), we are left with 9 different variants.
That's 9 variants, with 76 methods, and approximately 150 trait impls in the
stdlib. That is a big API surface, and that's not even considering all the
different combinations of iterators.</p>
Iterator</code> is probably the single-most complex trait in the language. It is a
junction in the language where every effect, auto-trait, and lifetime feature
ends up intersecting. And unlike similar junctions like the Fn</code>-family of
traits; the Iterator</code> trait is stable, broadly used, and has a lot of
combinators. Meaning it both has a broad scope, and stringent
backwards-compatibility guarantees we need to maintain.</p>
At the same time Iterator</code> is also not that special either. It's a pretty easy
trait to write by hand after all. The way I think of it is mainly as a canary for
language-wide shortcomings. Iterator</code> is for example not unique in its
requirement for stable addresses: we want to be able to guarantee this for
arbitrary types and to be able to use this with arbitrary interfaces. I believe
that the question to ask here should be: what is stopping us from using
address-sensitive types with arbitrary types and arbitrary interfaces? If we can
answer that, not only will we have an answer for Iterator</code> - we will have
solved it for all other interfaces we did not consciously anticipate would want
to interact with this 8</a></sup>.</p>
^{8</sup>
This is adjacent to "known unknowns" vs "unknown unknowns" - we should not just cater to cases we can anticipate, but also to cases we cannot. And that requires analyzing patterns and thinking in systems.</p>
</div>
In this post I've done my best to show by-example which limitations the
Iterator</code> trait has today, and how each variant can overcome those. And while I
believe that we should try and lift those limitations over time, I don't think
anyone is too keen on us minting 9 new variations on the core::iter</code> submodule.
Nor the thousand-plus possible combinations of those submodules (yay
combinatorics). The only feasible approach I see to navigating the problem space
is by extending, rather than replacing, the Iterator</code> trait. Here is my current
thinking for how we might be able to extend Iterator</code> to support all iterator
variants:</p>}
base trait</strong>: default, already supported</li>
dyn-compatible</strong>: default, already supported</li>
bounded</strong>: sub-trait, already supported</li>
fused</strong>: sub-trait, already supported</li>
thread-safe</strong>: auto-trait, already supported</li>
seeking</strong>: sub-trait</li>
compile-time</strong>: effect polymorphism (const</code>) 9</a></sup></li>
lending</strong>: 'move</code> lifetime 10</a></sup></li>
with return value</strong>: unsure 11</a></sup></li>
with next argument</strong>: default value + optional/variadic arg</li>
short-circuiting</strong>: effect polymorphism (try</code>)</li>
address-sensitive</strong>: auto-trait</li>
guaranteeing destruct</strong>: auto-trait</li>
async</strong>: effect polymorphism (async</code>)</li>
concurrent</strong>: new trait variant(s)</li>
</ul>
^{9</sup>
the const</code> effect itself is already polymorphic over the comptime effect, since const fn</code> means: "a function that can be executed either at comptime or runtime". Out of all the effect variants, const</code> is most likely to happen in the near-term.</p>
</div>}^{10</sup>
What we want to express is that we have an associated type which MAY take a lifetime, not MUST take a lifetime. That way we can pass a type by value where a type would otherwise be expected to be passed by-reference. This is different from both 'static</code> lifetimes and &own</code> references.</p>
</div>}^11</sup>I tried answering how to add return values to the Iterator</code> trait a year ago in my post Iterator as an Alias</a>, but I came up short. As I mentioned earlier in the post: combining "return with value" and "may short-circuit with error" seems tricky. Maybe there is a combination here I'm missing / we can special-case something. But I haven't seen it yet.</p>
</div>
When evolving the language, I believe we entire job is to balance feature
development with the management of complexity. If done right, over time the
language should not only become more capable, but also simpler and easier to
extend. To quote something TC (T-Lang) said in a recent conversation: "We should
get better at getting better every year". 12</a></sup></p>
As we think about how we wish to overcome the challenges presented by this post,
it is my sincere hope that we will increasingly start thinking of ways to solve
classes of problems that just happen to show up with Iterator</code> first. While at
the same time looking for opportunities to ship features sooner, without
blocking ourselves on supporting all of the use cases straight away.</p>
^{12</sup>
I recently remarked to TC that I've started to think about governance and language evolution as being about the derivative of the language and project. Rather than measuring the direct outcomes, we measure the process that produces those outcomes. TC remarked that what we should really care about is the double derivative. Not only should we improve our outcomes over time; the processes that produce those outcomes should improve over time as well. Or put differently: we should get better at getting better every year! I love this quote and I wanted y'all to read it too.</p>
</div>}

Musings on iterator trait names
Mon, 20 Jan 2025 00:00:00 +0000
At the end of my last
post</a> I mentioned that one
of the main issues with the IntoIterator</code> trait is that it’s kind of a pain to
write. I wasn’t around when it was first introduced, but it’s not hard to tell
that the original authors intended for Iterator</code> to be the primary interface
with IntoIterator</code> being an additional convenience.</p>
This didn’t quite turn out to be the case though, and it’s common practice to
use IntoIterator</code> in both bounds and impls. In the Rust 2024 edition we’re
changing Rust’s range type to implement IntoIterator</code> rather than Iterator</code>.</del> 1</a></sup>
And for example in Swift the equivalent trait to IntoIterator</code> (Sequence</code>) is
the primary interface used for iteration. With the interface equivalent to
Iterator</code> (IteratorProtocol</code>) having a much harder to use name.</p>
^{1</sup>
Thanks to Lukas Wirth for pointing out that the range type change
didn't end up making the cut for the edition. It's been a couple of months since
I checked, and it seems it was removed for this edition. My understanding is
that this change is still desired, and might make it in for a future edition.</p>
</div>
So if not Iterator</code>, what name could we use? Well, recently I wrote a
little library called Iterate</code> that tries to answer that question. Let me walk
you through it.</p>
Note: this post is intended to be a public exploration, not a concrete
proposal. It's a starting point, asking: "... what if?" I'm a firm advocate for
sharing ideas in public, especially if they're not yet fully fleshed out. There
are a lot of good reasons to do that, but above all else: I think it's fun!</em></p>}verbs, nouns, and traits</h2>
In Rust most interfaces use verbs as their names. To read bytes from a stream
you use the trait named Read</code>. To write bytes you use Write</code>. To debug
something you use Debug</code>. And to calculate numbers you can use Add</code>, Mul</code>
(multiply), or Sub</code> (subtract). Most traits in the Rust stdlib are used to
perform concrete operations with, and the convention is to use verbs for that.</p>
The stdlib does have one particularly interesting pairing in the form of Hash</code>
(verb) and Hasher</code> (noun). From the documentation: "Types implementing Hash</code>
are able to be hashed with an instance of Hasher</code>." Or put differently: the
trait Hash</code> represents the operation</em> and the trait Hasher</code> represents the
state</em>.</p>
/// A hashable type.
</span>pub trait </span>Hash {
</span>    </span>fn </span>hash</span><H: Hasher>(&</span>self</span>, </span>state</span>: &</span>mut</span> H);
</span>}
</span>
</span>/// Represents state that is changed while hashing data.
</span>pub trait </span>Hasher {
</span>    </span>fn </span>finish</span>(&</span>self</span>) -> </span>u64</span>;
</span>    </span>fn </span>write</span>(&</span>mut </span>self</span>, </span>bytes</span>: &[</span>u8</span>]);
</span>}
</span></code></pre>
A verb for iteration</h2>
What the trait IntoIterator</code> really represents is: "An iterable type". And the
trait Iterator</code> can be reasonably described as: "The state that is changed
while iterating over items". The verb/noun split present in Hash</code>/Hasher</code>
feels like it easily applies to iteration too.</p>
If Iterator</code> is the noun that represents the iteration state, what is the verb
that represents the capability? The obvious choice would be Iterate</code>. Which I
think ends up working out somewhat nicely. To iterate over items you implement
Iterate</code>, which provides you with a stateful Iterator</code>.</p>
/// An iterable type.
</span>pub trait </span>Iterate {
</span>    </span>type </span>Item;
</span>    </span>type </span>Iterator: Iterator<Item = </span>Self::</span>Item>;
</span>    </span>fn </span>iterate</span>(</span>self</span>) -> </span>Self::</span>Iterator;
</span>}
</span>
</span>/// Represents state that is changed while iterating.
</span>pub trait </span>Iterator {
</span>    </span>type </span>Item;
</span>    </span>fn </span>next</span>(&</span>mut </span>self</span>) -> Option<</span>Self::</span>Item>;
</span>}
</span></code></pre>
With our goal to make IntoIterator</code> less jarring to use in interfaces, the name
Iterate</code> doesn't seem half-bad. And it neatly follows the existing verb/noun
split pairing we're already using elsewhere in the stdlib.</p>
Collecting items</h2>
People familiar with Rust's stdlib will be quick to note that Iterator</code> and
IntoIterator</code> are not the only iteration traits in use. We also have
FromIterator</code> that functions as the inverse of IntoIterator</code>. Where one exists
to convert types to iterators, the other exists to convert iterators back to
types. The latter is typically used via the Iterator::collect</code> function.</p>
But IntoIterator</code> has a less-known but equally useful sibling: Extend</code>. Where
IntoIterator</code> collects items into new instances of types, the Extend</code> trait is
used to collect items into existing instances of types. It would feel pretty
weird to rename IntoIterator</code> to Iterate</code>, but then keep FromIterator</code> as-is.
What if instead of anchoring FromIterator</code> as a dual to IntoIterator</code>, we
instead treated it as a sibling to Extend</code>. The obvious verb for this would be
Collect</code>:</p>
/// Creates a collection with the contents of an iterator.
</span>pub trait </span>Collect<A>: Sized {
</span>    </span>fn </span>collect</span><T>(</span>iter</span>: T) -> </span>Self
</span>    </span>where
</span>        T: Iterate<Item = A>;
</span>}
</span></code></pre>
It's interesting to note that the type T</code> in FromIterator</code> is bound by
IntoIterator</code> rather than Iterator</code>. Being able to use T: Iterate</code> as a bound
here definitely feels a little nicer. And speaking of nicer: this would also
make the provided Iterator::collect</code> and Iterator::collect_into</code> methods feel
a little better:</p>
/// Represents state that is changed while iterating.
</span>pub trait </span>Iterator {
</span>    </span>type </span>Item;
</span>    </span>fn </span>next</span>(&</span>mut </span>self</span>) -> Option<</span>Self::</span>Item>;
</span>
</span>    </span>/// Create a collection with the contents of this iterator.
</span>    </span>fn </span>collect</span><C>(</span>self</span>) -> C
</span>    </span>where
</span>        C: Collect<</span>Self::</span>Item>,
</span>        </span>Self</span>: Sized,
</span>
</span>    </span>/// Extend a collection with the contents of this iterator.
</span>    fn collect_into<E>(self, collection: &</span>mut</span> E) -> &</span>mut</span> E
</span>    </span>where
</span>        E: Extend<</span>Self::</span>Item>,
</span>        </span>Self</span>: Sized;
</span>}
</span></code></pre>
I don't think this looks half-bad. And honestly: it might also be more
consistent overall, as traits representing other effects don't have an
equivalent to FromIterator</code>. The Future</code> trait only has IntoFuture</code>, and
variations on that in the ecosystem like
Race</code></a>.
No longer having a trait called FromIterator</code> would help remove some confusion.</p>
Async</h2>
I guess we've broached the async topic now, so I guess we might as well keep
going. We added the trait IntoFuture</code> to Rust in 2022 because we wanted an
equivalent to IntoIterator</code> but for the async effect. You can find some
motivating use cases for this in my async builders
(2019)</a> post. We chose the name
IntoFuture</code> because it matched the existing convention set by
IntoIterator/Iterator</code>.</p>
We already have Try</code> for fallibility, we just discussed using Iterate</code> for
iteration, what would the verb-based trait name be for asynchrony? The obvious
choice would be something like Await</code>, as that is the name of the operation:</p>
trait </span>Await {
</span>    </span>type </span>Output;
</span>    </span>type </span>Future = Future<Output = </span>Self::</span>Output>;
</span>    </span>fn </span>await</span>(</span>self</span>) -> </span>Self::</span>Future;
</span>}
</span></code></pre>
This however runs into one major limitation: await</code> is a reserved keyword,
which means we can't use it as the name of the method. Which means I'm not
actually sure what this trait should be called. With iterators we're lucky that
we don't have any shortage of related words: loop</code>, iterate</code>, generate</code>,
while</code>, sequence</code> and so on. With async we're a little shorter on words. If
anyone has any good ideas for verbs to use here, I'd love to hear suggestions!</p>
Conclusion</h2>
TLDR: I really wouldn't mind Iterate</code> being the main interface name for
iteration in Rust. That seems like it would be a step up from writing
IntoIterator</code> in bounds everywhere. Just by changing a name, without the need
for any special new language features.</p>
Now for whether we should make this change:.. maybe? I honestly don't know. It's
not just a matter of introducing a simple trait alias either: the method names and
associated types are also different and we can't alias those. And I'm not
particularly keen for Rust to start dabbling in additional trait hierarchies
here either. Iteration is complex enough as it is, more super-traits are not
going to make things any simpler here.</p>
//! Renaming and aliasing the trait is not
</span>//! enough, the method names and associated
</span>//! type names would need to be aliased too.
</span>pub trait </span>Iterate { .. }
</span>pub trait </span>IntoIterator = Iterate;
</span></code></pre>
So I think the only way this rename would actually make sense to follow through
on is if the process to make changes like these would make that change easy. I
don't believe it is today, but I definitely believe we should want it to become
easy in the future. It would be nice if we could freely rename traits, methods,
and maybe even types across editions without causing any breakage.</p>
Either way though: I had a lot of fun writing this post. If you want to try the
Iterate</code> trait yourself today to get a better feel for it - check out the
iterate-trait</code></a> crate. It
has everything I've described in this post, as well as iterator combinators like
map</code>. Probably don't use if for anything serious, but definitely go having fun
with it.</p>

The gen auto-trait problem
Mon, 13 Jan 2025 00:00:00 +0000
One of the open questions surrounding the unstable gen {}</code> feature is whether
it should return Iterator</code> or IntoIterator</code>. People have had a feeling there
might be good reasons for it to return IntoIterator</code>, but couldn't necessarily
articulate why</em>. Which is why it was included in the "unresolved questions"
section on the gen blocks RFC.</p>
Because I'd like to see gen {}</code> blocks stabilize sooner, I figured it would be
worth spending some time looking into this question and see whether there are
any reasons to pick one over the other. And I have found what I believe to be a
fairly annoying issue with gen</code> returning Iterator </code> that I've started calling
the gen auto-trait problem. In this post I'll walk through what this problem is,
as well as how gen</code> returning IntoIterator</code> would prevent it. So without
further ado, let's dive in!</p>
Leaking auto-traits from gen bodies</h2>
The issue I've found has to do with auto-trait impls on reified gen {}</code>
instances. Take the thread::spawn</code> API: it takes a closure which returns a type
T</code>. If you haven't seen its definition before, here it is:</p>
pub fn </span>spawn</span><F, T>(</span>f</span>: F) -> JoinHandle<T>
</span>where
</span>    F: FnOnce() -> T + Send + </span>'static</span>,
</span>    T: Send + </span>'static</span>,
</span></code></pre>
This signature states that the closure F</code> and return type T</code> must be Send</code>.
But what it doesn't state is that all local values created inside the closure
F</code> must be Send</code> too - which is common in the async world. Now let's create an
iterator using a gen {}</code> block that we try and send across threads. We can create an iterator that yields a single u32:</p>
let</span> iter = gen {
</span>    </span>yield</span> </span>12</span>u32</span>;
</span>};
</span></code></pre>
Now if we try and pass this to thread::spawn</code>, there are no problems. Our
iterator implements + Send</code> and things will work as expected:</p>
// ✅ Ok
</span>thread::spawn(</span>move </span>|| {
</span>    </span>for</span> num in iter {
</span>        </span>println</span>("</span>{num}</span>");
</span>    }
</span>}).</span>unwrap</span>();
</span></code></pre>
Now to show you the problem, let's try this again but this time with a gen {}</code>
block that internally creates a std::rc::Rc</code>. This type is !Send</code>, which means
that any type that holds it will also be !Send</code>. That means that iter</code> in our
example here is now no longer thread-safe:</p>
let</span> iter = gen { </span>// `-> impl Iterator + !Send`
</span>    </span>let</span> rc = Rc::new(...);
</span>    </span>yield</span> </span>12</span>u32</span>;
</span>    rc.</span>do_something</span>();
</span>};
</span></code></pre>
That means if we try and send it across threads we'll get a compiler error:</p>
// ❌ cannot send a `!Send` type across threads
</span>thread::spawn(</span>move </span>|| {   </span>// ← `iter` is moved
</span>    </span>for</span> num in iter {     </span>// ← `iter` is used
</span>        </span>println</span>("</span>{num}</span>");
</span>    }
</span>}).</span>unwrap</span>();
</span></code></pre>
And that right there is the problem. Even though iterators are lazy and the Rc</code>
in our gen {}</code> block isn't actually constructed until iteration begins on the
new thread, the generator block needs to reserve space for it internally and so
it inherits the !Send</code> restriction. This leads to incompatibilities where
locals defined entirely within gen {}</code> itself end up affecting the
public-facing API. This ends up being very subtle and tricky to debug, unless
you're already familiar with the generator desugaring.</p>
Preventing the leakage</h2>
Solving the gen's auto-trait problem is not that hard. What we want is for the
!Send</code> fields in the generator to not show up in the generated type until we
are ready to start iterating over it. That sounds a little scary, but in
practice all we have to do is for gen {}</code> to start returning an impl IntoIterator</code> rather than an impl Iterator</code>. The actual Iterator</code> will still be !Send</code>, but our type IntoIterator</code> will be Send</code>:</p>
let</span> iter = gen { </span>// `-> impl IntoIterator + Send`
</span>    </span>let</span> rc = Rc::new(...);
</span>    </span>yield</span> </span>12</span>u32</span>;
</span>    rc.</span>do_something</span>();
</span>};
</span></code></pre>
Since our value iter</code> implements Send</code> we can now happily pass it across
thread bounds. And our code continues to operate as expected, as for..in</code>
operates on IntoIterator</code> which is implemented for all Iterator</code>:</p>
// ✅ Ok
</span>thread::spawn(</span>move </span>|| {   </span>// ← `iter` is moved
</span>    </span>for</span> num in iter {     </span>// ← `iter` is used
</span>        </span>println</span>("</span>{num}</span>");
</span>    }
</span>}).</span>unwrap</span>();
</span></code></pre>
If you think about it, from a theoretical perspective this makes a lot of sense
too. We can think of for..in</code> as our way of handling the iteration effect,
which expects an IntoIterator</code>. gen {}</code> is the dual of that, used to create
new instances of the iteration effect. It's not at all strange for it to return
the same trait that for..in</code> expects. With the added bonus that it doesn't leak auto-traits from impl bodies to the type's signature.</p>
What about other effects and auto-traits?</h2>
This issue primarily affects effects which make use of the generator
transform</em>. That is: iteration and async. In theory I believe that, yes, async {}</code> should probably return IntoFuture</code> rather than Future</code>. In WG Async we regularly see issues that have to do with auto-traits leaking from inside function bodies out into the type's signatures. If async</code> (2019) would have returned IntoFuture</code> (2022) rather than Future</code> (2019) that certainly seems like it could have helped. Though there would be a higher barrier to make that change today now that things are already stable.</p>
On the trait side this doesn't just affect Send</code> either: it applies to all
auto-traits, present and future. Though Send</code> is by far the most common trait
people will experience issues with today, to a lesser extent this also already
applies to Sync</code>. And in the future possibly also auto-traits like
Freeze</code></a>,
Leak</code></a>, and
Move</code></a>. Though this
shouldn't be the main motivator, preventing potential future issues is not
worthless either.</p>
Conclusion</h2>
Admittedly the worst part of gen {}</code> returning IntoIterator</code> is that the name
of the trait kind of sucks. IntoIterator</code> sounds auxiliary to Iterator</code>, and
so it feels a little wrong to make it the thing we return from gen {}</code>. But on top of that: that's a lot of characters to write.</p>
I wonder what would have happened if we'd taken an approach more similar to
Swift. Swift has
Sequence</code></a> which is
like Rust's IntoIterator</code> and
IteratorProtocol</code></a>
which is like Rust's Iterator</code>. The primary interface people are expected to
use is short and memorable. While the secondary interface isn't meant to be
directly used, and so it has a much longer and less memorable name. As we're
increasingly thinking of IntoIterator</code> as the primary interface for async
iteration, maybe we'll want to revisit the trait naming scheme in the future.</p>
In conclusion: it seems like a good idea to prevent auto-traits from leaking from
gen bodies when reified. This is the kind of issue that if we can prevent we
should prevent, as it can be both difficult to diagnose and annoying to work
around. Having auto-trait leakage be a non-issue for generator blocks seems
worthwhile.</p>

What are temporal and spatial memory safety?
Sun, 15 Dec 2024 00:00:00 +0000
The lovely folks working on security over at Google have recently been writing
about "temporal (memory)
safety"</a>
and "spatial (memory)
safety"</a>.
When I first saw these terms it took me a minute to figure out what they meant,
as searching for it online didn't yield immediate answers. So I figured it might
be helpful to write it down for others to find:</p>

Spatial memory safety:</strong> describes violations like out-of-bounds access. Say you have a vec of 10 items, it's undefined behavior if you try and read from the memory location of the non-existent 11th item. You can think of these as violations that have to do with memory regions</em> (space).</li>
Temporal memory safety:</strong> describes violations like use-after-free. Say you have a type that has been de-initialized already ("dropped" in Rust), it's undefined behavior to then try and read from any of its fields. You can think of these as violations that have to do with the ordering of memory operations</em> (time).</li>
</ul>
My only qualm with these terms is that I've occasionally seen people drop the
"memory" qualifier. I think this is part of what makes these terms confusing:
there are other many more safety properties 1</a></sup> involving layout or
ordering that can be modeled that don't have anything to do with memory safety. Skimming the entries
on Temporal Logic</a> or
TLA+</a> should make that clear enough 2</a></sup>.</p>
^1</sup>When modeling, the term "safety" refers to the absence of a defined negative property. That's in contrast to "liveness" which refers to the presence of a defined positive property. Read more</a>.</p>
</div>
^{2</sup>
If you prefer a concrete example: imagine two concurrent writers to the same TCP socket speaking HTTP. Without careful coordination between the two this will lead to issues involving ordering of operations</em> (temporal safety) that may lead to data loss, data corruption, and so on - but without ever involving involving any memory safety violations.</p>
</div>
My ask then is that when discussing spatial memory safety and temporal
memory safety to refer to them as such in full. Don't try and shorten them by
dropping the "memory" qualifier. Because when discussing formally modeled
properties it certainly pays to be specific about what it is that you're trying to
guarantee.</p>}

Why `Pin` is a part of trait signatures (and why that's a problem)
Tue, 15 Oct 2024 00:00:00 +0000
I've been wondering for a little while why the Future::poll</code> methods takes
self: Pin<&mut Self></code> in its signature. I assumed there must have been a good
reason, but when I asked my fellow WG Async members nobody seemed to know off
hand why that was exactly. Or maybe they did, and I just had some trouble
following. Either way, I think I've figured it out, and I want to spell it out
for posterity so that others can follow too.</p>
Why Pin</code> is part of method signatures</h2>
Take for example a type MyType</code> and a trait MyTrait</code>. We can write an
implementation of MyTrait</code> which is only available when MyType</code> is pinned:</p>
trait </span>MyTrait {
</span>    </span>fn </span>my_method</span>(&</span>mut </span>self</span>) {}
</span>}
</span>struct </span>MyType;
</span>impl</span><T> MyTrait </span>for </span>Pin<&</span>mut</span> MyType> {}
</span></code></pre>
Inside of functions we can even write bounds for this. Shout out here to Eric
Holk who showed me that apparently the left-hand side of trait bounds
can contain arbitrary types - not only generics or even types that are part of
the function signature. I had no idea.</p>
With that we can express that we're taking some type T</code> by-value, and once we
pin that value it will implement MyTrait</code>:</p>
fn </span>my_function</span><T>(</span>input</span>: T)
</span>where
</span>    </span>for</span><</span>'a</span>> Pin<&</span>'a mut</span> T>: MyTrait,
</span>{
</span>    </span>let</span> pinned_input = pin!(input);
</span>}
</span></code></pre>
Inside of MyTrait::my_method</code>, the type of self</code> will be &mut Pin<&mut Self></code>. That's not the same as the owned type Pin<&mut Self></code>, but luckily we
can convert that into an owned type by calling
Pin::as_mut</code></a>.
The docs contain a big explainer why it's safe here to go from a mutable
reference to an owned instance, which intuitively goes against Rust's ownership
rules.</p>
But what happens now if rather than writing a generic type T</code> with a where</code>
clause, we instead want to use an impl trait in associated position (APIT). We
might want to write something like this:</p>
// how do we express those same bounds here?
</span>fn </span>my_function</span><T>(</span>input</span>: impl ???) {
</span>    </span>let</span> pinned_input = pin!(input);
</span>}
</span></code></pre>
But we have no way to express that exact bound. Unlike regular generics, APITs
can't express the left-hand side of the bound (lvalue), they can only name the
right-hand side (rvalue). This becomes even more pronounced when we try and use
impl Trait</code> in return position (RPTIT).</p>
Take for example a function that returns some type T</code>. Using concrete trait
bounds we can express that this returns a type which when pinned implements
MyTrait</code>:</p>
fn </span>my_function</span><T>() -> T
</span>where
</span>    </span>for</span><</span>'a</span>> Pin<&</span>'a mut</span> T>: MyTrait,
</span>{
</span>    MyType {}
</span>}
</span></code></pre>
But if we then try and express that same function using RPTIT, we lose the
ability to express that bound. The only solution to express an -> impl Trait</code>
which exposes functionality when it's pinned, is to make Pin</code> directly part of
the signature on the methods, and not implement the trait for a Pin<&mut Type></code>:</p>
trait </span>MyTrait {
</span>    </span>fn </span>my_method</span>(</span>self</span>: Pin<&</span>mut Self</span>>) {} </span>// ← note the signature of self here
</span>}
</span>struct </span>MyType;
</span>impl </span>MyTrait </span>for </span>MyType {} </span>// ← no longer implemented for `Pin<&mut MyType>`
</span></code></pre>
And now all of a sudden we can express -> impl MyTrait</code> whose methods can only
be called when MyType</code> is pinned. With Unpin</code> being the opt-out for types
where that is not the case.</p>
fn </span>my_function</span>() -> impl MyTrait { </span>// Can be either pinned or not!
</span>    MyType {}
</span>}
</span></code></pre>
Implications</h2>
Concretely what this means is that if you want to have a trait that wants to
work with pinned values and work with all language features like normal, you
have to use self: Pin<&mut Self></code> part of the method signature. Maybe that's
not a big deal for new traits, but it has implications for every existing trait
in the stdlib.</p>
Take for example the Iterator</code> trait. We can't just impl Iterator for Pin<&mut T></code> and expect RPTIT to work. Instead the expected route here seems like it
should be to introduce a new trait PinnedIterator</code> that takes self: Pin<&mut T></code>. This is a backwards-incompatibility shared by all existing traits in the
stdlib except for Future</code>, which already takes self: Pin<&mut Self></code>. That's a
pretty big limitation, and something that's worth factoring into discussions about the viability of Pin</code> beyond Future</code>. For Iterator</code> it means we would want to mint at least the following variants:</p>
// iterator
</span>trait </span>Iterator {
</span>    </span>type </span>Item;
</span>    </span>fn </span>next</span>(&</span>mut </span>self</span>) -> Option<Item>;
</span>}
</span>
</span>// address-sensitive iterator
</span>trait </span>PinnedIterator {
</span>    </span>type </span>Item;
</span>    </span>fn </span>next</span>(</span>self</span>: Pin<&</span>mut</span> self>) -> Option<Item>;
</span>}
</span></code></pre>
To run through some more implications of this: if we want to Rust users to be
able to declare address-sensitive types in Rust, then the most likely path now
is a duplication of the traits in the std::io</code> submodule taking a shape similar
to this:</p>
mod </span>io {
</span>    </span>pub trait </span>Read { ... }
</span>    </span>pub trait </span>PinnedRead { ... }
</span>    </span>pub trait </span>Write { ... }
</span>    </span>pub trait </span>PinnedWrite { ... }
</span>    </span>pub trait </span>Seek { ... }
</span>    </span>pub trait </span>PinnedSeek { ... }
</span>    </span>pub trait </span>BufRead { ... }
</span>    </span>pub trait </span>PinnedBufRead { ... }
</span>}
</span></code></pre>
Pinning, like async</code> and try</code>, is a combinatorial property of traits which leads to an exponential amount of duplication. Lucky for us duplicating traits is not the only possible
path we can take: some form of polymorphism for existing interfaces over Pin</code> seems possible too - as long as we're willing to change our formulation of it. Which was what lead me to formulate
my design for the Move</code> auto-trait</a>, which is composable just like e.g. Send</code> and Sync</code>.</p>
Conclusion</h2>
I want to quickly shout out my fellow WG Async folks. We've been talking about
this a bunch, and it's been helpful working through this. Even if it's taken me a couple of months to actually get around to posting this. Any mistakes in this
post however are definitely my own.</p>
In this post I mainly wanted to articulate why Future</code> takes self: Pin<&mut Self></code> rather than &mut self</code> and rely on impl Future for Pin<&mut T></code>. I
think I've found a good reason for that, and it has again to do with the left-
and right-hand side of bounds. For me this also confirms my hypothesis that any
design for generalized self-referential types needs to be able to capture the following:</p>

The ability to mark a type as being immovable</li>
The ability to transition types from being movable to immovable</li>
The ability to construct immovable types in-place</li>
The ability to extend existing interfaces with immovability</li>
The ability to describe self-referential lifetimes</li>
The ability to safely initialize self-references without Option</code></li>
</ol>
This post specifically covered the fourth requirement: the ability to extend
existing interfaces with immovability. Addressing that can take the form of duplication of
interfaces (e.g. Iterator</code> vs PinnedIterator</code>), or composition via
(auto-)traits (e.g. Move</code>). Other methods might be possible too, and I would
encourage people with ideas to share them.</p>
PoignardAzur has independently
described</a> why and how
pinned types need to be able to be constructed in-place. Their post showed
examples of both the third and sixth requirements in the list. They introduced a
form of emplacement via the -> pin Type</code> notation. This is similar to the more
general -> super Type</code></a>
notation I introduced in my post, which I adapted from Mara's super let</code>
post</a>.</p>
I hope this post helps at least partially clarify why Pin</code> needs to be
part of interfaces. As well as helped spell out some of the logical consequences
once we consider how it interacts with the stdlib's strict
backwards-compatibility requirements. Because I believe we can and should do
better than duplicating entire interfaces over the axis of immovability.</p>

Solving my RSI issues
Mon, 23 Sep 2024 00:00:00 +0000
For the past decade I've struggled with wrist injuries on and off. It's a common
occupational injury for us keyboard users, and after my most recent bout this
year I decided it was time to actually do something about it. Permanently. I
like systemic solutions - they tickle a good part of my brain. And I figured I
could apply some of that to my keyboard habits too.</p>
In this post I want to share a little about what my journey to actually fix my
wrist issues. Not to be prescriptive for others, but to be descriptive about
what I'm doing. Perhaps it'll be useful for others, but mainly because it's fun
for me to talk about. I've been making little ASCII cheat sheets for myself, and
basically I just wanted to have an excuse to post them publicly.</p>
Breaking up writing sessions</h2>
The way I've dealt with my RSI issues for the past eight years is by taking
repeated breaks. I've done this using the Time Out
app</a>
on macOS, and Workrave</a> on Windows and Linux. Both
applications bring up a modal pop-up on a set schedule. I have it set to create
a short 30-second break every 10 minutes or so, and a 10-minute break every 50
minutes.</p>
While this does not make typing itself any less strenuous, it does break up some
of the repetition - which ends up helping a lot. Taking breaks can be a little
frustrating though, as it can break up flow. But it's better to have a lot of
small breaks than longer-lasting injuries. Shout out to Jacob Groundwater for
suggesting this back in 2016 - it's helped me a lot over the years.</p>
Mobility</h2>
Another thing I've been doing is mobility issues, specifically for my wrist. I
worked with a physio from 2018 to 2020, and I have a stretch routine that helps
with flexibility and mobility. I have to admit I've not been keeping that up,
but as symptoms flare back up, it's something that I know I can apply when
needed.</p>
Where taking breaks helps break up the intensity of the strain, physical
exercises help both with symptom relief and building resiliency. It's not
perfect, but it's yet another thing I've found that helps me. I'm a little
hesitant to provide instructions since I'm not a physio myself though, and it's
definitely possible to hurt yourself if you don't do it right.</p>
Choosing a keyboard layout</h2>
I started taking regular breaks again in June this year, and it's not helped me
as well as it did before. I assume it has something to do with being older now,
and recovery just taking longer than it used to. Though I learned how to
touch-type when I was in my early teens, I know that using QWERTY layouts aren't
particularly good for you. It requires a lot of extra movement in fingers, which
leads to a lot of extra strain.</p>
Because I hope to continue to use computers in some capacity for the next fifty
odd years, I figured re-learning something I only learned twenty years ago would
be well-worth it in the long run. So time to bite the bullet, and learn a new
layout. But which one?</p>
I'm lucky that many of my friends are incredibly geeky</del>knowledgeable, and I
figured one of them would probably have a good suggestion for what to learn. A
quick check online suggested that Colemak
Mod-DH</a> would probably be a good choice.
So I checked with my friend Sy Brand, who thought it was a good layout - but
they suggested something better: Canary</a>.</p>
Canary is a fun little layout based on Colemak Mod-DH but optimizes for
something called "rolls". In keyboard terminology a "roll" is a sequence of keys
you can type with one hand. A layout which has "high rolls" means it's a layout
that has many of those sequences. Here is what the layout looks like on a
regular keyboard:</p>
w l y p k z x o u ; [ ] \
</span> c r s t b f n e i a '
</span>  j v d g q m h / , .
</span></code></pre>
Rolls in QWERTY are fairly uncommon. But with Canary a lot of words incorporate
rolls. This might be easier to show rather than explain, so here are some words
that incorporate rolls on a QWERTY layout: asimo</code>, werk</code>, wookie</code>. Do you
notice how typing those feels kind of flowy? Now imagine nearly all words feel
like that. Creating that feeling without causing strain is what the Canary
layout was specifically designed for.</p>
Learning a new keyboard layout</h2>
Once I'd decided to switch to a new keyboard layout, there was a big "draw the
rest of the owl" step to actually being able to use it. When I was a kid my mom
paid for me to take after-school classes for a number of weeks to learn how to
touch-type. That was 20 years ago though, and I somehow doubt there are
in-person classes teaching the Canary layout. So what to do?</p>
It turns out there are lots of great options now. I've been using
keybr.com</a> which is free to use, supports various
layouts in-browser, and provides different guided trainings to learn how to
touch-type with new layouts. I've been using it daily for the past six weeks,
and honestly it does everything I could hope for.</p>
To complement keybr I've also started using ngram
type</a> which allows you to train
"N-grams". These are sequences of characters which are often grouped together.
Those high rolls I was talking about earlier? N-grams are the sequences they
refer to.</p>
Finally I've also occasionally been using monkeytype</a>
which is more focused on leaderboards and good looks. A lot of keyboard
influencers (they exist) use this to show off how well they can type or how
good their keyboard sounds. I haven't found it particularly useful to learn new
layouts with, but it's been fun to play with!</p>
On QWERTY I can comfortably type around 90 words per minute (wpm). When learning
Canary it took me a few days to get to 10 wpm (unusable), two weeks to get to
20wpm (better, but still unusable) and about a month to get to 40 wpm (usable
but slow). I spent about 10 total hours in keybr, averaging about 30 mins of
training a day. I have no sense for whether that's fast or slow, but I was
positively surprised that I could rebuild 20 years of muscle memory in a matter
of weeks, with a fairly modest time investment.</p>
Using vim movements on a new layout</h2>
One thin I knew would be an issue and actually turned out to be an issue was
editing code on a new layout. I use vim bindings in all of my editors, and vim
motions are just part of me now. I asked Sy about how they switched over to a
new layout when they did and deal with this, and well, they didn't. Good on them
for not using vim bindings; sucks to be me.</p>
This turned out to be a bit of a stumbling block: while I don't exclusively</em>
write code, I do still write a lot of it. And that means that I need to be able
to move around an editor. I mean, I know there are other vim movements and I use
those too. But heck, not being able to use the base vim motions to navigate?
Yeah, I wasn't going to give that up.</p>
I looked into changing key bindings in VS Code, and tried figuring out what
others had done - and there didn't seem to be any good solutions. Except one:
Miryoku</a>.</p>
Keyboard layers</h2>
One of the issues with QWERTY is that you move around your fingers a lot, which
causes strain over time. Miryoku is one attempt to define a layout which
minimizes the amount of movement your fingers need to make. It's based on two principles:</p>

Each finger should at most ever switch one key over</li>
The strongest fingers (thumbs, index fingers) should do the most work</li>
</ul>
The way it achieves this is by defining an entire layout which can be accessed
from the base position. The entire system is layer-based (modal), with each
layer being accessible by pressing down a specific key with your thumb. Numbers,
symbols, and navigation each get their own layer. And by putting the arrow keys
on the position of the vim bindings, it makes it possible to restore vim-like
navigation just by pressing down a single key. Here is what the neutral (base)
layer looks like:</p>
w l y p b     z f o u '      ¹esc   ⁴ret
</span>c r s t g     m n e i a      ²spc   ⁵bsp
</span>q j v d k     x h / , .      ³tab   ⁶del
</span>    ¹ ² ³     ⁴ ⁵ ⁶
</span></code></pre>
But once we press and hold the space key (key 2) with our thumb, we engage the
navigation layer and the arrow keys become available to us. It's not quite the same position as with vim - but you can configure it that way if you like too. Here's what it looks like by default:</p>
- - - - -     - - - - -
</span>- - - - -     - ← ↓ ↑ →
</span>- - - - -     - - - - -
</span>    ◼︎ - -     - - -    
</span></code></pre>
I thought this was neat and probably the right solution, but I wasn't sure how
to engage the thumb keys on a regular keyboard. The space bar is in the way, and
there is no thumb cluster for us to press. One solution is to shift your fingers
one row up, and remap the keyboard to use the XQCBNM keys as our thumb keys.
miryoku_kmonad</a> is a software
package for macOS and Linux which allows you to do exactly that. But I tried and
failed to get that to work reliably (or well: I lacked the patience to finish
debugging it), so I went to find a different solution.</p>
Hardware</h2>
So far my plan was to keep using my existing hardware. I like my 60% Vortex
Cypher keyboard, and I figured I'd start by switching keyboard layouts and see
how far I could get. I tried that and got stuck after a while - so I figured I
might as well commit to the bit and invest in hardware.</p>
I ended up going picking up a pre-soldered Chocify
keyboard</a> from
Beekeeb</a>.
This is a 36-key split keyboard which comes with Miryoku pre-installed. I got
the bluetooth-based version with the screen which cost a little extra. I did
make a mistake by choosing the "Choc Blue" switches which are different from
Cherry MX blue switches, so I've got a new set of tactile switches coming in soon (Kaihl Sunset).</p>
Custom keyboards are pretty involved, but well worth it. Some things I didn't know before I got started but had to learn on the way:</p>

international shipping of batteries is difficult - if you need bespoke batteries for a hardware project in Europe, Amazon is actually one of the better options.</li>
the Chocify board uses the nice!nano micro-controller. The way you flash this is by double-clicking a button that turns it into USB drive mode, and then you can just drag and drop new firmware on it from your computer</li>
you can solder cables to connect them. Though you want to heat-shrink them
after. I didn't actually end up doing this, but got help from my local coffee
shop with this hah (if you're reading this: thanks Valdi!)</li>
</ul>
Picking up hardware was something I didn't want to do unless strictly necessary -
but I think it was the right choice. I've recently also started experiencing
shoulder pain after long typing sessions, and I've noticed that having a split ergo keyboard actually helps a lot with that too.</p>
I'm fairly confident this little 36-key keyboard I have is going to become my
new daily driver once I get the hang of it. And while that isn't the case yet, that moment is creeping up on me soon.</p>
Learning a new keyboard layout (again)</h2>
Oh before I close out, it turns out that the Canary keyboard layout comes in two
variants: staggered and ortho. I didn't realize the two were different, so I
just pressed "canary" on keybr and started learning. It turns out that was the
staggered layout:</p>
w l y p k z x o u ; [ ] \
</span> c r s t b f n e i a '
</span>  j v d g q m h / , .
</span></code></pre>
On a 30% keyboard that layout ends up looking like this:</p>
w l y p k     z x o u ; 
</span>c r s t b     f n e i a
</span>j v d g q     m h / , .
</span></code></pre>
The layout the Canary authors designed for keyboards like this is the "ortho"
layout which looks like this. It's very similar, but not the same:</p>
w l y p b     z f o u '
</span>c r s t g     m n e i a
</span>q j v d k     x h / , .
</span></code></pre>
Oops, I should have paid more attention before I got started. Though in all
fairness: I did start learning Canary under the assumption I'd stick with my 60%
staggered keyboard. So it makes sense I didn't immediately jump to the ortho
layout. Though I probably should have thought to start learning the ortho layout
as soon as I put in the hardware order - which was a few weeks ago now.</p>
Conclusion</h2>
Now that I have my keyboard, have the firmware flashed and batteries installed,
this is where I'm currently at in my journey: re-learning to type on the Canary
Ortho layout. I think this is fine, because when switching to the Miryoku layout
I need to re-learn where all the symbols are, so it's not like I can hit the
ground running on my new keyboard anyway.</p>
I'm still waiting on my new key switches to arrive which should make typing on
my new keyboard a lot easier (45g rather than 20g actuation force - MX blues
have 50g actuation). But I can already tell that Canary + Miryoku + Chocify
causes a lot less strain on my hands than QWERTY does.</p>
I'm not going to say that people should follow what I'm doing. I don't know your
situation, and maybe just getting an ergo QWERTY keyboard or taking more breaks
might be enough for your needs. But for me? I've really enjoyed going down the
rabbit hole of learning a new keyboard layout, getting a new keyboard, and
picking up various things along the way.</p>
So has all of this solved my RSI issues? No - I'm not yet daily driving my new
keyboard, so I'm not yet experiencing the benefits. But I can tell the difference in strain I
experience when I switch to the split keyboard and new keyboard layout. Now that nearly
everything is in place to switch over, I can't wait to start daily driving it.
I'm definitely taking the most involved path to solving this, but I think that
this might actually be the answer to fully fix the source of my injuries.</p>
This post was a little break from my usual writing about type systems and language
design. If you'd like to learn more about ergonomic keyboard layouts, I really enjoyed this talk by Mattia Dal Ben: 34 keys is all you need: an ergonomic keyboard journey</a>.</p>
</p>
Here's what my keyboard practice looked like this morning. I think it's cute.</em></p>

Further simplifying self-referential types for Rust
Mon, 08 Jul 2024 00:00:00 +0000
In my last post</a> I
discussed how we might be able to introduce ergonomic self-referential types (SRTs) to
Rust, mostly by introducing features we know we want in some form anyway. The
features listed were:</p>

Some form of 'unsafe</code> and 'self</code> lifetimes.</li>
A safe out-pointer notation for Rust (super let</code> / -> super Type</code>).</li>
A way to introduce out-pointers without breaking backwards-compat.</li>
A new Move</code> auto-trait that can be used to mark types as immovable (!Move</code>).</li>
View types which make it possible to safely initialize self-referential types.</li>
</ol>
This post was received quite well, and I thought the discussion which followed
was quite interesting. I learned about a number of things that I think would
help refine the design further, and I thought it would be good to write it up.</p>
Not all self-referential types are !Move</code></h2>
Niko Matsakis pointed out that not all self-referential types are necessarily
!Move</code>. For example: if the data being referenced is heap-allocated, then the
type doesn't actually have to be !Move</code>. When writing protocol parsers, it's
actually fairly common to read data into a heap-allocated type first. It seems
likely that a fair number of self-referential types don't actually also need to
be !Move</code> or any concept of Move</code> at all to function. Which also means we
don't need some form of super let</code> / -> super Type</code> to construct types
in-place.</p>
If we just want to enable self-references for heap-allocated types, then all we
need for that is a way to initialize them (view types) and an ability to
describe the self-lifetimes ('unsafe</code> as a minimum). That should give us a good
idea of what we can prioritize to begin enabling a limited form of
self-references.</p>
The 'self</code> lifetime is insufficient</h2>
Speaking of lifetimes, Mattieum pointed
out</a> that
'self</code> likely wasn't going to be enough. 'self</code> points to an entire struct,
which ends up being too coarse to be practical. Instead we need to be able to
point to individual fields to describe lifetimes.</p>
Apparently Niko has also come up with a feature for this, in the form of
lifetimes based on
places</a>.
Rather than having abstract lifetimes like the 'a</code> that we use to link to
values with, it would be nicer if references just always had implicit, unique
lifetime names. With access to that, we should rewrite the motivating example
from our last post from being based on 'self</code>:</p>
struct </span>GivePatsFuture {
</span>    </span>resume_from</span>: GivePatsState,
</span>    </span>data</span>: Option<String>,
</span>    </span>name</span>: Option<&</span>'self str</span>>,        </span>// ← Note the `'self` lifetime
</span>}
</span></code></pre>
To instead be based on paths:</p>
struct </span>GiveManyPatsFuture {
</span>    </span>resume_from</span>: GivePatsState,
</span>    </span>first</span>: Option<String>,
</span>    </span>data</span>: Option<&</span>'self</span>.</span>first </span>str</span>>, </span>// ← Note the `'self.first` lifetime
</span>}
</span></code></pre>
This might not seem like it's that important in this example; but once we
introduce mutability things quickly spiral. And not introducing a magic 'self</code>
in favor of always requiring 'self.field</code> seems like it would generally be
better. And that requires having lifetimes which can be based on places, which
seems like a great idea regardless.</p>
Automatic referential stability</h2>
Earlier in this post we established that we don't actually need to encode
!Move</code> for self-referential types which store their values on the heap. That's
not all self-referential types - but does describe a fair number of them. Now
what if we didn't need to encode !Move</code> for almost the entire remaining set of
self-referential types?</p>
If that sounds like move constructors, you'd be right - but with a catch! Unlike
the Relocate</code> trait I described in my last post, DoveOfHope
noted</a> that we
might not even need that to work. After all: if the compiler already knows that
we're pointing to a field contained within the struct - can't the compiler make
sure to update the pointers when we try and move the structure?</p>
I was skeptical about the possibility of this until I read about place-based
lifetimes. With that it seems like we would actually have enough granularity to
know how to update which fields when they are moved. In terms of cost: that's
just updating the value of a pointer on move - which is effectively free. And it
would rid us almost entirely of needing to encode !Move</code>.</p>
The only cases not covered by this would be 'unsafe</code> references or actual
*const T</code> / *mut T</code> pointers to stack data. The compiler doesn't actually know
what those point to, and so cannot update them on move. For that some form of
Relocate</code> trait actually seems like it would be useful to have. But that's
something that wouldn't need to be added straight away either.</p>
Raw pointer operations and automatic referential stability</h2>
This section was added after publication, on 2024-07-08.</em></p>
While it should be possible for the compiler to guarantee that the codegen is
correct for e.g. mem::swap</code></a>,
we can't make those same guarantees for raw pointer operations such as
ptr::swap</code></a>. And because
existing structures may be freely using those operations internally, that means
on-stack self-referential types can't just be made to work without any caveats
the way we can for on-heap SRTs. That's indeed a problem, and I want to thank
The_8472</a> for
pointing this out.</p>
I was really hoping to be able to avoid extra bounds, so that on-stack SRTs
could match the experience of on-heap SRTs. But that doesn't seem possible, so
perhaps some minimum amount of bounds which we can flip to being set by default
(like Sized</code>) over an edition might be enough to do the trick here. I'm
currently thinking of something like:</p>

Introduce a new auto marker trait Transfer</code> to complement Relocate</code>, as a dual
to Rust's Destruct</code></a>
/ Drop</code></a> system. Transfer</code>
is the name of the bound people would use, Relocate</code> provides the hooks to
extend the Transfer</code> system.</li>
All types with 'self</code> lifetimes automatically implement Transfer</code>.</li>
Only bounds which include + Transfer</code> can take impl Transfer</code> types.</li>
All relevant raw pointer move operations must uphold additional safety
invariants of what to do with impl Transfer</code> types.</li>
We gradually update the stdlib to support + Transfer</code> in all bounds.</li>
Over some edition we make opt-out rather than opt-in (T: Transfer</code> → T: ?Transfer</code>).</li>
</ol>
auto </span>trait </span>Transfer {}
</span>
</span>trait </span>Relocate { ... }
</span></code></pre>
I was really hoping we could avoid something like this. And it does put into
question whether this is actually simpler than immovable types. But the_8472 is
quite right that this is an issue, and so we need to address it. Luckily we've
already done something like this before with const</code>. And I don't think that
this is something we can generalize. I'll write more about this at some later
date.</p>
Relocate</code> should probably take &own self</code></h2>
Now, even if we don't expect people to need to write their own pointer update
logic, basically like ever, it's still something that should be provided. And
when we do, we should encode it correctly.  Nadrieril very helpfully pointed out
that the &mut self</code> bound on the Relocate</code> trait might not actually be what we
want - because we're not just borrowing a value - we actually want to destruct
it. Instead they informed me about the work done towards &own</code>, which would give access to something called: "owned references".</p>
Daniel Henry-Mantilla is the author of the stackbox</code>
crate</a> as well as the main person responsible for the
lifetime extension system behind the pin!</code> macro in the stdlib. A while back he
shared a very helpful writeup about
&own</code></a>.
The core of the idea is that we should decouple the concepts of: "Where is the
data concretely stored?"</em> from: "Who logically owns the data?"</em> Resulting in
the idea of having a reference which doesn't just provide temporary</em> unique
access - but can take permanent</em> unique access. In his post, Daniel helpfully
provides the following table:</p>
</th> Semantics for T</code></th> For the backing allocation</th></tr></thead>

&T</code></td> Shared access</td> Borrowed</td></tr>
&mut T</code></td> Exclusive access</td> Borrowed</td></tr>
&own T</code></td> Owned access  
(drop responsibility)</td> Borrowed</td></tr>
</tbody></table>
Applying this to our post, we would use this to change the trait Relocate</code> from
taking &mut self</code>, which temporarily takes exclusive access of a type - but
can't actually drop the type:</p>
trait </span>Relocate {
</span>    </span>fn </span>relocate</span>(&</span>mut </span>self</span>) -> super </span>Self</span>;
</span>}
</span></code></pre>
To instead take &own</code>, which takes permanent exclusive access of a type, and
can actually drop the type:</p>
trait </span>Relocate {
</span>    </span>fn </span>relocate</span>(&own </span>self</span>) -> super </span>Self</span>;
</span>}
</span></code></pre>
edit 2024-07-08: This example was added later</em>. To explain what &own</code> solves,
let's take a look at the Relocate</code> example impl from our last post. In it we
say the following:</p>

We're making one sketchy assumption here: we need to be able to take the owned
data from self, without running into the issue where the data can't be moved
because it is already borrowed from self.</p>
</blockquote>
struct </span>Cat {
</span>    </span>data</span>: String,
</span>    </span>name</span>: &</span>'self str</span>,
</span>}
</span>impl </span>Cat {
</span>    </span>fn </span>new</span>(</span>data</span>: String) -> super </span>Self </span>{ ... }
</span>}
</span>impl </span>Relocate </span>for </span>Cat {
</span>    </span>fn </span>relocate</span>(&</span>mut </span>self</span>) -> super </span>Self </span>{
</span>        </span>let mut</span> data = String::new();                   </span>// ← dummy type, does not allocate
</span>        mem::swap(&</span>mut </span>self</span>.data, &</span>mut</span> data);           </span>// ← take owned data
</span>        </span>super let</span> cat = Cat { data };                   </span>// ← construct new instance
</span>        cat.name = cat.data.</span>split</span>(' ').</span>next</span>().</span>unwrap</span>(); </span>// ← create self-ref
</span>        cat                                             </span>// ← return new instance
</span>    }
</span>}
</span></code></pre>
What &own</code> gives us is a way to correctly encode the semantics here. Because
the type is not moved we can't actually move by-value. But logically we do still
want to claim unique ownership of the value so we can destruct the type and move
individual fields. That's kind of the way moving Box</code> by-value works too, but
rather than having the allocation on the heap, the allocation can be anywhere.
With that we could rewrite the rather sketchy mem::swap</code> code above into more normal-looking destructuring + initialization instead:</p>
struct </span>Cat {
</span>    </span>data</span>: String,
</span>    </span>name</span>: &</span>'self str</span>,
</span>}
</span>impl </span>Cat {
</span>    </span>fn </span>new</span>(</span>data</span>: String) -> super </span>Self </span>{ ... }
</span>}
</span>impl </span>Relocate </span>for </span>Cat {
</span>    </span>fn </span>relocate</span>(&</span>mut </span>self</span>) -> super </span>Self </span>{
</span>        </span>let Self </span>{ data, .. } = </span>self</span>;                   </span>// ← destruct `self`
</span>        </span>super let</span> cat = Cat { data };                   </span>// ← construct new instance
</span>        cat.name = cat.data.</span>split</span>(' ').</span>next</span>().</span>unwrap</span>(); </span>// ← create self-ref
</span>        cat                                             </span>// ← return new instance
</span>    }
</span>}
</span></code></pre>
Now, because this actually does need to construct types in fixed memory
locations this trait would need to have some form of -> super Self</code> syntax.
After all: this would be the one place where it would still be needed. For
anyone interested in keeping up to date with &own</code>, here is the Rust issue for it</a> (which also happens to have
been filed by Niko).</p>
Motivating example, reworked again</h2>
With this in mind, we can rework the motivating example from the last post once
again. To refresh everyone's memory, this is the high-level async/.await</code>-based
Rust code we would:</p>
async </span>fn </span>give_pats</span>() {
</span>    </span>let</span> data = "</span>chashu tuna</span>".</span>to_string</span>();
</span>    </span>let</span> name = data.</span>split</span>(' ').</span>take</span>().</span>unwrap</span>();
</span>    </span>pat_cat</span>(&name).await;
</span>    println!("</span>patted </span>{name}</span>");
</span>} 
</span>
</span>async </span>fn </span>main</span>() {
</span>    </span>give_pats</span>().await;
</span>}
</span></code></pre>
And using the updates in this post we can proceed to desugar it. This time
around without the need for any reference to Move</code> or in-place construction,
thanks to path-based lifetimes and the compiler automatically preserving
referential stability:</p>
enum </span>GivePatsState {
</span>    Created,
</span>    Suspend1,
</span>    Complete,
</span>}
</span>
</span>struct </span>GivePatsFuture {
</span>    </span>resume_from</span>: GivePatsState,
</span>    </span>data</span>: Option<String>,
</span>    </span>name</span>: Option<&</span>'self</span>.</span>data </span>str</span>>, </span>// ← Note the `'self.data` lifetime
</span>}
</span>impl </span>GivePatsFuture {   </span>// ← No in-place construction needed
</span>    </span>fn </span>new</span>() -> </span>Self </span>{
</span>        </span>Self </span>{
</span>            resume_from: GivePatsState::Created,
</span>            data: None,
</span>            name: None
</span>        }
</span>    }
</span>}
</span>impl </span>Future </span>for </span>GivePatsFuture {
</span>    </span>type </span>Output = ();
</span>    </span>fn </span>poll</span>(&</span>mut </span>self</span>, </span>cx</span>: &</span>mut </span>Context<'_>)  </span>// ← No `Pin` needed
</span>        -> Poll<</span>Self::</span>Output> { ... }
</span>}
</span></code></pre>
This is significantly simpler than what we had before in its definition. And even the actual desugaring of the invocation ends up being simpler: no longer needing the intermediate IntoFuture</code> constructor to guarantee in-place construction.</p>
let</span> into_future = GivePatsFuture::new();
</span>let mut</span> future = into_future.</span>into_future</span>(); </span>// ← No `pin!` needed
</span>loop </span>{
</span>    </span>match</span> future.</span>poll</span>(&</span>mut</span> current_context) {
</span>        Poll::Ready(ready) => </span>break</span> ready,
</span>        Poll::Pending => </span>yield</span> Poll::Pending,
</span>    }
</span>}
</span></code></pre>
All that's needed for this is for the compiler to update the addresses of
self-pointers on move. That's a little bit of extra codegen whenever a value is
moved - rather than just a bitwise copy, it also needs to update pointer values.
But it seems quite doable to implement, should actually perform really well, and
most importantly: users would rarely if ever need to think about it. Writing
&'self.field</code> would always just work.</p>
When immovable types are still needed</h2>
I don't want to discount the idea of immovable types entirely though. There are
definitely benefits to having types which cannot be moved. Especially when
working with FFI structures that require immovability. Or some high-performance
data structures which use lots of self-references to stack-based data which
would be too expensive to update. Use cases certainly exist, but they're going
to be fairly niche. For example: Rust for Linux uses immovable types for their
intrusive linked lists - and I think those probably need some form of
immovability to actually work.</p>
However, if the compiler doesn't require immovable types to provide
self-references, then immovable types suddenly go from being load-bearing to
instead becoming something closer to an optimization. It's likely still worth
adding them since they certainly are more efficient. But if we do it right,
adding immovable types will be backwards-compatible, and would be something we
can introduce later as an optimization.</p>
When it comes to whether async {}</code> should return impl Future</code> or impl IntoFuture</code>: I think the answer really should be impl IntoFuture</code>. In the 2024
edition we're changing range syntax (0..12</code>) from returning Iterator</code> to
returning IntoIterator</code></a>.
This matches Swift's behavior, where 0..12</code> returns
Sequence</code></a> rather
than IteratorProtocol</code></a>. I think this is a good indication that async {}</code> and gen {}</code> probably also should return impl Into*</code>  traits rather than their respective traits.</p>
Conclusion</h2>
I like it when something I write is discussed and I end up picking up on other
relevant work. I think to enable self-referential types I'm now definitely
favoring a form of built-in pointer updates as part of the language over
immovable types (edit: maybe I spoke too soon</a>). However, if we do want immovable types - I think my last post
provides a coherent and user-friendly design to get us there.</p>
There are a fairly large number of dependencies if we want to implement a
complete story for self-referential types. Luckily we can implement features one
at a time, enabling increasingly more expressive forms of self-referential
types. In terms of importance: some form of 'unsafe</code> seems like a good starting
point. Followed by place-based lifetimes. View types seem useful, but aren't in
the critical path since we can work around phased initialization using an option
dance. Here's a graph of all features and their dependencies.</p>
</p>
Breaking down the features like has actually further reinforced my perception
that this all seems quite doable. 'unsafe</code> doesn't seem like it's that far out.
And Niko has sounded somewhat serious about path-based lifetimes and view types.
We'll have to see how quickly those will actually end up being developed in
practice - but having laid this all out like this, I'm feeling somewhat
optimistic!</p>

Ergonomic Self-Referential Types for Rust
Mon, 01 Jul 2024 00:00:00 +0000
I've been thinking a little about self-referential types recently, and while
it's technically possible</em> to write them today using Pin</code> (limits apply),
they're not at all convenient. So what would it take to make it convenient?
Well, as far as I can tell there are four components involved in making it work:</p>

The ability to write 'self</code> lifetimes.</li>
The ability to construct types from functions in fixed memory locations.</li>
A way to mark types as "immovable" in the type system.</li>
The ability to safely initialize self-references in structs without going through an option-dance.</li>
</ol>
It's only once we have all four of these components that writing
self-referential types can become accessible to most regular Rust programmers.
And that seems important, because as we've seen with async {}</code> and Future</code>:
once you start writing sufficiently complex state machines, being able to track
references into data becomes incredibly useful.</p>
Speaking of async</code> and Future</code>: in this post we'll be using that as a
motivating example for how these features can work together. Because if it seems
realistic that we can make that a case as complex as that work, other simpler
cases should probably work too.</p>
Oh and before we dive in, I want to give a massive shout-out to Eric Holk. We've
spent several hours working through the type-system implications of !Move</code>
together, and worked through a number of edge cases and issues. I can't be
solely credited for the ideas in this post. However any  mistakes in this post are mine, and I’m not claiming to speak for the both of us.</p>
Disclaimer: This post is not a fully-formed design. It is an early exploration
of how several features could work together to solve a broader problem. My goal
is primarily to narrow down the design space to a tangible list of features
which can be progressively implemented, and share it with the broader Rust
community for feedback. I'm not on the lang team, nor do I speak for the lang
team.</em></p>
Motivating example</h2>
Let's take async {}</code> and Future</code> as our examples here. When we borrow local
variables in an async {}</code> block across .await</code> points, the resulting state
machine will store both the concrete value and a reference to that value in the
same state machine struct. That state machine is what we call
self-referential</em>, because it has a reference</em> which points to something in
self</em>. And because references are pointers to concrete memory addresses, there
are challenges around ensuring they are never invalidated as that would result
in undefined behavior. Let's look at an example async function:</p>
async </span>fn </span>give_pats</span>() {
</span>    </span>let</span> data = "</span>chashu tuna</span>".</span>to_string</span>();       </span>// ← Owned value declared
</span>    </span>let</span> name = data.</span>split</span>(' ').</span>next</span>().</span>unwrap</span>(); </span>// ← Obtain a reference
</span>    </span>pat_cat</span>(&name).await;                       </span>// ← `.await` point here
</span>    println!("</span>patted </span>{name}</span>");                  </span>// ← Reference used here
</span>} 
</span>
</span>async </span>fn </span>main</span>() {
</span>    </span>give_pats</span>().await; </span>// Calls the `give_pats` function.
</span>}
</span></code></pre>
This is a pretty simple program, but the idea should come across well enough: we
declare an owned value in-line, we call an .await</code> function, and later on we
reference the owned value again. This keeps a reference live across an .await</code>
point, and that requires self-referential types. We can desugar this to a future
state machine like so</a>:</p>
enum </span>GivePatsState {
</span>    Created,       </span>// ← Marks our future has been created
</span>    Suspend1,      </span>// ← Marks the first `.await` point
</span>    Complete,      </span>// ← Marks the future is now done
</span>}
</span>
</span>struct </span>GivePatsFuture {
</span>    </span>resume_from</span>: GivePatsState,
</span>    </span>data</span>: Option<String>,
</span>    </span>name</span>: Option<&</span>str</span>>,  </span>// ← Note the lack of a lifetime here
</span>}
</span>
</span>impl </span>GivePatsFuture {
</span>    </span>fn </span>new</span>() -> </span>Self </span>{
</span>        </span>Self </span>{
</span>            resume_from: GivePatsState::Created,
</span>            data: None,
</span>            name: None,
</span>        }
</span>    }
</span>}
</span>
</span>impl </span>Future </span>for </span>GivePatsFuture {
</span>    </span>type </span>Output = ();
</span>    </span>fn </span>poll</span>(</span>self</span>: Pin<&</span>mut Self</span>>, </span>cx</span>: &</span>mut </span>Context<'_>) -> Poll<</span>Self::</span>Output> { ... }
</span>}
</span></code></pre>
The lifetime of GivePatsFuture::name</code> is unknown, mainly because we can't name
it. And because the desugaring happens in the compiler, it doesn't have to name
the lifetime either. We'll talk more about that later in this post. Because this
generates a self-referential state machine, this future will need to be fixed
in-place using Pin</code> first. Once pinned, the Future::poll</code> method can be called
in a loop until the future yields Ready</code>. The desugaring for that will look
something like this</a>:</p>
let mut</span> future = IntoFuture::into_future(GivePatsFuture::new());
</span>let mut</span> pinned = </span>unsafe </span>{ Pin::new_unchecked(&</span>mut</span> future) };
</span>loop </span>{
</span>    </span>match</span> pinned.</span>poll</span>(&</span>mut</span> current_context) {
</span>        Poll::Ready(ready) => </span>break</span> ready,
</span>        Poll::Pending => </span>yield</span> Poll::Pending,
</span>    }
</span>}
</span></code></pre>
And finally, just for reference, here is what the traits we're using look like
today. The main bit that's interesting here for the purpose of this post is that
Future</code> takes a Pin<&mut Self></code>, which we'll be explaining how it can be
replaced with a simpler system throughout the remainder of this post.</p>
pub trait </span>IntoFuture {
</span>    </span>type </span>Output;
</span>    </span>type </span>IntoFuture: Future<Output = </span>Self::</span>Output>;
</span>    </span>fn </span>into_future</span>(</span>self</span>) -> </span>Self::</span>IntoFuture;
</span>}
</span>
</span>pub trait </span>Future {
</span>    </span>type </span>Output;
</span>    </span>fn </span>poll</span>(</span>self</span>: Pin<&</span>mut Self</span>>, </span>cx</span>: &</span>mut </span>Context<'_>) -> Poll<</span>Self::</span>Output>;
</span>}
</span></code></pre>
Now that we've taken a look at how self-referential futures are desugared  by
the compiler today, let's take a look at how we can incrementally replace
it with a safe, user-constructible system.</p>
Self-referential lifetimes</h2>
In our motivating example we showed the GivePatsFuture</code> which has the name</code>
field that points at the data</code> field. It's a clearly a reference, but it does not carry any lifetime:</p>
struct </span>GivePatsFuture {
</span>    </span>resume_from</span>: GivePatsState,
</span>    </span>data</span>: Option<String>,
</span>    </span>name</span>: Option<&</span>str</span>>, </span>// ← Note the lack of a lifetime
</span>}
</span></code></pre>
The reason this doesn't have a lifetime is not inherent</em>, it's because we can't
actually name the lifetime here. It's not 'static</code> because it isn't valid for
the remainder of the program. In the compiler today I believe we can just omit
the lifetime because the codegen happens after the lifetimes have already been
checked. But say we wanted to write this by hand today as-is; we would need a
concept for an "unchecked lifetime", something like this:</p>
struct </span>GivePatsFuture {
</span>    </span>resume_from</span>: GivePatsState,
</span>    </span>data</span>: Option<String>,
</span>    </span>name</span>: Option<&</span>'unsafe str</span>>, </span>// ← Unchecked lifetime
</span>}
</span></code></pre>
Being able to write a lifetime that isn't checked by the compiler would be the
first-click stop to enable writing self-referential structs by hand. It would
just require a lot of unsafe</code> and anxiety to get right. But at least it would
be possible. I believe folks on T-compiler are already working on adding this,
which seems like a great idea.</p>
But even better would be if we could describe checked</em> lifetimes here. What we
actually want to write here is a lifetime which is valid for the duration of the
value - and it would always be guaranteed to be valid. Adding this lifetime
would come with additional constraints we'll get into later in this post (e.g.
the type wouldn't be able to move), but what we really want is to be able to
write something like this:</p>
struct </span>GivePatsFuture {
</span>    </span>resume_from</span>: GivePatsState,
</span>    </span>data</span>: Option<String>,
</span>    </span>name</span>: Option<&</span>'self str</span>>, </span>// ← Valid for the duration of `Self`
</span>}
</span></code></pre>
To sidetrack slightly; adding a named 'self</code> lifetime could also allow us to remove the
where Self: 'a</code> boilerplate when using lifetimes in generic associated types. If you've ever
worked with lifetimes in associated types, then you'll likely have run into the "missing required bounds" error. Niko suggested in this
issue</a> using 'self</code> as one possible solution for the bounds boilerplate. I
think its meaning would be slightly different than when used with
self-references? But I don't believe using this would be ambiguous
either. And all in all I think it looks rather neat:</p>
// A lending iterator trait as we have to write it today:
</span>trait </span>LendingIterator {
</span>    </span>type </span>Item<</span>'a</span>>
</span>    </span>where
</span>        </span>Self</span>: </span>'a</span>;
</span>    </span>fn </span>next</span>(&</span>mut </span>self</span>) -> </span>Self::</span>Item<'_>;
</span>}
</span>
</span>// A lending iterator trait if we had `'self`:
</span>trait </span>LendingIterator {
</span>    </span>type </span>Item<</span>'self</span>>;
</span>    </span>fn </span>next</span>(&</span>mut </span>self</span>) -> </span>Self::</span>Item<'_>;
</span>}
</span></code></pre>
Constructing types in-place</h2>
In order for 'self</code> to be valid, we have to promise our value won't move in
memory. And before we can promise a value won't move, we have to first construct
it somewhere we can be sure it can stay and not be moved further. Looking at our
motivating example, the way we achieve this using Pin</code> is a little goofy.
Here's that same example again:</p>
impl </span>GivePatsFuture {
</span>    </span>fn </span>new</span>() -> </span>Self </span>{
</span>        </span>Self </span>{
</span>            resume_from: GivePatsState::Created,
</span>            data: None,
</span>            name: None,
</span>        }
</span>    }
</span>}
</span>
</span>async </span>fn </span>main</span>() {
</span>    </span>let mut</span> future = IntoFuture::into_future(GivePatsFuture::new());
</span>    </span>let mut</span> pinned = </span>unsafe </span>{ Pin::new_unchecked(&</span>mut</span> future) };
</span>    </span>loop </span>{
</span>        </span>match</span> pinned.</span>poll</span>(&</span>mut</span> current_context) {
</span>            Poll::Ready(ready) => </span>break</span> ready,
</span>            Poll::Pending => </span>yield</span> Poll::Pending,
</span>        }
</span>    }
</span>}
</span></code></pre>
In this example we're seeing GivePatsFuture</code> being constructed inside of the
new</code> function, be moved out of that, and only then pinned in-place using
Pin::new_unchecked</code>. Even if GivePatsFuture: !Unpin</code>, the Unpin</code> trait only
affects types once they are held inside of a Pin</code> structure. And we can't just
return Pin</code> from new</code>, because the function's stack frames are discarded the
moment the function returns.</p>
It would be better if we enabled types to describe how they can construct
themselves in-place. That means no more external Pin::new_unchecked</code> calls; but
exclusively internally provided constructors. This enables us to make
self-referential types entirely self-contained, with internally provided
constructors replacing the external pin dance. Here's how we could rewrite GivePatsFuture::new</code> to use an internal constructor instead:</p>
impl </span>GivePatsFuture {
</span>    </span>fn </span>new</span>(</span>slot</span>: Pin<&</span>mut </span>MaybeUninit<</span>Self</span>>>) {
</span>        </span>let</span> slot = </span>unsafe </span>{ slot.</span>get_unchecked_mut</span>() };
</span>        </span>let</span> this: </span>*mut Self </span>= slot.</span>as_mut_ptr</span>();
</span>        </span>unsafe </span>{ 
</span>           addr_of_mut!((*this).resume_from).</span>write</span>(GivePatsState::Created);
</span>           addr_of_mut!((*this).data).</span>write</span>(None);
</span>           addr_of_mut!((*this).name).</span>write</span>(None);
</span>        };
</span>    }
</span>}
</span></code></pre>
If you don't like this: that's understandable. I don't think anyone does. But
bear with me; we're on a little didactic sausage-making journey here together.
I'm sorry about the sights; let's quickly move on.</p>
In a recent blog
post</a>
I posited we might be able to think of parameters like these as "spicy return".
James Munns pointed out that in C++ this feature has a name: out-pointers. And
Jack Huey made an interesting connection between this and the super let</code>
design</a>. Just so we don't have to look at a
pile of unsafe</code> code again, let's pretend we can combine these into something
coherent:</p>
impl </span>GivePatsFuture {
</span>    </span>fn </span>new</span>() -> super Pin<&</span>'super mut Self</span>> {
</span>        pin!(</span>Self </span>{ </span>// just pretend this works
</span>            resume_from: GivePatsState::Created,
</span>            data: None,
</span>            name: None,
</span>        })
</span>    }
</span>}
</span></code></pre>
I know, I know - I'm showing syntax here. I'm personally meh</em> about the way it
looks and we'll talk more later about how we can improve this, but I hope we can
all agree that the function body itself is approximately 400% more legible than the pile of unsafe</code> we
were working with earlier.  We'll also get to how we can entirely remove Pin</code>
from the signature, so please don't get too hung up on that either.</p>
I'm asking you to play along here for a second, and pretend we might be able to
do something like this for now, so we can get to the part later on this post
where we can actually fix it. In maybe state this more clearly: this post is
less about proposing concrete designs for a problem, but more about how we can
tease apart the problem of "immovable types" into separate features we can
tackle independently from one another.</p>
Converting into immovable types</h2>
Alright, so we have an idea of how we might be able to construct immovable types
in-place, provided as a constructor defined on a type. Now while that's nice,
we've also lost an important property with that: whenever GivePatsFuture</code> is
constructed, it needs to have a fixed place in memory. Where before we could
freely move it around until we started .await</code>ing it.</p>
One of the main reasons why async is useful is because it enables ad-hoc
concurrent execution</a>. That means
we want to be able to take futures and pass them to concurrency operations to
enable concurrency through composition. We can't move futures which have a fixed
location in memory, so we need a brief moment where futures can be moved before
they're ready to be kept in place and polled to completion.</p>
The way Pin</code> works with this today is that a type can be !Unpin</code> - but that
only becomes relevant once it's placed inside of a Pin</code> structure. With futures
that typically doesn't happen until it begins being polled, usually via
.await</code>, and so we get the liberty of moving !Unpin</code> futures around until we
start .await</code>ing them. That's why !Unpin</code> doesn't mark: "A type which cannot
be moved", it marks: "A type which cannot be moved once it has been pinned".
This is definitely confusing, so don't worry if it's hard to follow.</p>
fn </span>foo</span><T: Unpin>(</span>t</span>: &</span>mut</span> T);   </span>// Type is not pinned, type can be moved.
</span>fn </span>foo</span><T: </span>!</span>Unpin>(t: &</span>mut</span> T);  </span>// Type is not pinned, type can be moved.
</span>fn </span>foo</span><T: Unpin>(</span>t</span>: Pin<&</span>mut</span> T>);  </span>// Type is pinned, type can be moved.
</span>fn </span>foo</span><T: </span>!</span>Unpin>(t: Pin<&</span>mut</span> T>); </span>// Type is pinned, type can't be moved.
</span></code></pre>
If we want "immovability" to be unconditionally part of a type, we can't
make it behave the same way Unpin</code> does. Instead it seems better to separate
the movable / immovable requirements into two separate types. We first construct
a type which can be freely moved around - and once we're ready to drive it to
completion, we convert it to a type which is immovable and we begin calling
that. This maps perfectly to the separation between IntoFuture</code> and Future</code> we
already use.</p>
Let's take a look at our first example again, but modify it slightly. What I'm
proposing here is that rather than give_pats</code> returning an impl Future</code>, it
should instead return an impl IntoFuture</code>. This type is not pinned, and can be
freely moved around. It's only once we're ready to .await</code> it that we call
.into_future</code> to obtain the immovable future - and then we call that.</p>
struct </span>GivePatsFuture { ... } 
</span>impl </span>GivePatsFuture {
</span>    </span>fn </span>new</span>() -> super Pin<&</span>'super mut Self</span>> { ... } </span>// suspend belief pls
</span>}
</span>
</span>struct </span>GivePatsIntoFuture;
</span>impl </span>IntoFuture </span>for </span>GivePatsIntoFuture {
</span>    </span>type </span>Output = ();
</span>    </span>type </span>IntoFuture = GivePatsFuture;
</span>
</span>    </span>// We call the `Future::new` constructor which gives us a
</span>    </span>// `Pin<&'super GivePatsFuture>`, and then rather than writing
</span>    </span>// it into the current function's stack frame we write it in the
</span>    </span>// caller's stack frame.
</span>    </span>//
</span>    </span>// (keep belief suspended a little longer)
</span>    </span>fn </span>into_future</span>(</span>self</span>) -> super Pin<&</span>'super mut</span> GivePatsFuture> {
</span>        GivePatsFuture::new() </span>// create in caller's scope
</span>    }
</span>}
</span></code></pre>
Just like we can keep returning values from functions to pass them further up
the call stack, so should we be able to use out-pointers / emplacement / spicy
allocate in a stack frame further up the call stack. Though even if we didn't
support that out of the gate, we could probably in-line the
GivePatsFuture::new</code> into GivePatsIntoFuture::into_future</code> and things would
still work. And with that, our .await</code> desugaring could then look something like this:</p>
async </span>fn </span>main</span>() {
</span>    </span>let</span> into_future: GivePatsIntoFuture = </span>give_pats</span>();
</span>    </span>let mut</span> future: Pin<&</span>mut</span> GivePats> = GivePatsIntoFuture.</span>into_future</span>();
</span>    </span>loop </span>{
</span>        </span>match</span> future.</span>poll</span>(&</span>mut</span> current_context) {
</span>            Poll::Ready(ready) => </span>break</span> ready,
</span>            Poll::Pending => </span>yield</span> Poll::Pending,
</span>        }
</span>    }
</span>}
</span></code></pre>
To re-iterate why this section exists: we can get the same functionality Pin</code> +
Unpin</code> provide today by creating two separate types. One type which can be
freely moved around. And another type which once constructed will not move
locations in memory.</p>
So far the only framing of "immovable types" I've seen so far is a single types
which have both these properties - just like Unpin</code> does today. What I'm trying
to articulate here is that we can avoid that issue if we choose to create two
types instead, enabling one to construct the other, and make them provide
separate guarantees. I think that's a novel insight, and one I thought was important to spend some time on.</p>
Immovable types</h2>
Alright, I've been asking folks to suspend belief that we can in fact perform
in-place construction of a Pin<&mut Self></code> type somehow and that would all work
out the way we want it to. I'm not sure myself, but for the sake of the
narrative of this post it was easier if we just pretended we could for a second.</p>
The real solution here, of course, to get rid of Pin</code> entirely. Instead types
themselves should be able to communicate whether they have a stable memory
location or not. The simplest formulation for this would be to add a new
built-in auto-trait, Move</code>, which tells the compiler whether a type
can be moved or not.</p>
auto </span>trait </span>Move {}
</span></code></pre>
This is of course not a new idea: we've known about the possibility for Move</code>
since at least 2017. That's before I started working on Rust. There were some
staunch advocates for that in the Rust community in favor of Move</code>, but
ultimately that wasn't the design we ended up going
with</a>. I
think in hindsight most of will acknowledge that the downsides of Pin</code> are real
enough that revisiting Move</code> and working through its limitations seems like a
good idea 1</a></sup>. To explain what the Move</code> trait is: it would be a language-level trait which governs access to the following capabilities:</p>
^{1</sup>
For anyone looking to assign blame here or pull skeletons out of closets: please don't. The higher-order bit here is that we have Pin</code> today, it clearly doesn't work as well as was hoped at the time, and we'd like to replace it with something better. I think the most interesting thing to explore here is how we can move forward and do better.</p>
</div>}
The ability to be passed by-value into functions and types.</li>
The ability to be passed by mutable reference to mem::swap</code></a>, mem::take</code></a>, and mem::replace</code></a>.</li>
The ability to be used with any syntactic equivalents to the earlier points, such
as assigning to mutable references, closure captures, and so on.</li>
</ul>
Conversely, when a type implements !Move</code> they would not have access to any of
these capabilities - making it so they cannot be moved once they have a fixed
memory location. And by default we would assume in all bounds that types are
Move</code>, except for places that explicitly opt-out by using + ?Move</code>. Here are
examples of things that constitute moves:</p>
// # examples of moving
</span>
</span>// ## swapping two values
</span>let mut</span> x = </span>new_thing</span>();
</span>let mut</span> y = </span>new_thing</span>();
</span>swap</span>(&</span>mut</span> x, &</span>mut</span> y);
</span>
</span>// ## passing by value
</span>fn </span>foo</span><T>(</span>x</span>: T) {}
</span>let</span> x = </span>new_thing</span>();
</span>foo</span>(x);
</span>
</span>// ## returning a value
</span>fn </span>make_value</span>() -> Foo {
</span>    Foo {
</span>        x: </span>42
</span>    }
</span>}
</span>
</span>// ## `move` closure captures
</span>let</span> x = </span>new_thing</span>();
</span>thread::spawn(</span>move </span>|| {
</span>    </span>let</span> x = x;
</span>})
</span></code></pre>
And here are some things that do not constitute moves:</p>
// # things that are not moves
</span>
</span>// ## passing a reference
</span>fn </span>take_ref</span><T>(</span>x</span>: &T) {}
</span>let</span> x = </span>new_thing</span>();
</span>take_ref</span>(&x);
</span>
</span>// ## passing mutable references is also okay,
</span>//    but you have to be careful how you use it
</span>fn </span>take_mut_ref</span><T>(</span>x</span>: &</span>mut</span> T) {}
</span>let mut</span> x = </span>new_thing</span>();
</span>take_mut_ref</span>(&</span>mut</span> x);
</span></code></pre>
Passing types by-value will never</em> be compatible with !Move</code> types because that’s what a move is. Passing
types by-reference will always</em> be compatible with !Move</code> types because they are immutable 2</a></sup>. The only place with some ambiguity is when we work with
mutable references, as things like mem::swap</code> allow us to violate the
immovability guarantees.</p>
^{2</sup>
Yes yes, we'll get to internal mutability in a second here.</p>
</div>
If a function wants to take a mutable reference which may be immovable, they
will have to add + ?Move</code> to it. If a function does not use + ?Move</code> on their
mutable reference, then a !Move</code> type cannot be passed to it. In practice this
will work as follows:</p>}fn </span>meow</span><T>(</span>cat</span>: T);            </span>// by-value,   can't pass `!Move` values
</span>fn </span>meow</span><T>(</span>cat</span>: &T);           </span>// by-ref,     can pass `!Move` values
</span>fn </span>meow</span><T>(</span>cat</span>: &</span>mut</span> T);       </span>// by-mut-ref, can't pass `!Move` values
</span>fn </span>meow</span><T: </span>?</span>Move>(cat: &</span>mut</span> T) </span>// by-mut-ref, can pass `!Move` values
</span></code></pre>
By default all cat: &mut T</code> bounds would imply + Move</code>. And only where we
opt-in to + ?Move</code> could !Move</code> types be passed. In practice it seems likely
most places will probably be fine adding + ?Move</code>, since it's far more common
to write to a field of a mutable reference than it is to replace it whole-sale
using mem::swap</code>. Things like interior mutability are probably also largely
fine under these rules, since even if accesses go through shared references,
updating the values in the pointers will have to interact with the earlier rules
we've set out - and those are safe by default.</p>
To be entirely accurate we also have to consider internal mutability. That
allows us to mutate values through shared references - but only by being able to
conditionally convert it to &mut</code> references at runtime. Just because we allow
casting &T</code> to &mut T</code> at runtime, doesn't mean that the rules we've applied
to the system don't still work. Say we held an &mut T: !Move</code> inside of a
Mutex</a>. If we tried to
call the deref_mut</code>
method</a>,
we'd get a compile-error because that bound hasn't yet declared that T: ?Move</code>.
We could probably add that, but because it doesn't work by default we'd have an
opportunity to validate its soundness before adding it.</p>
Anyway, that's enough theory about how this should probably work for now. Let's
try and update our earlier example, replacing Pin</code> with !Move</code>. That should be
as simple as adding a !Move</code> impl on GivePatsFuture</code>.</p>
struct </span>GivePatsFuture {
</span>    </span>resume_from</span>: GivePatsState,
</span>    </span>data</span>: Option<String>,
</span>    </span>name</span>: Option<&</span>'self str</span>>,
</span>}
</span>impl </span>!Move for GivePatsFuture {}
</span></code></pre>
And once we have that, we can change our constructors to return super Self</code>
instead of super Pin<&'super mut Self></code>. We already know that emplacement using
something like super Self</code> (not actual notation) to write to fixed memory
locations seems plausible. All we then need to do is add an auto-trait which tells the type-system that further move operations aren't allowed.</p>
struct </span>GivePatsFuture { ... } 
</span>impl </span>!Move for GivePatsFuture {}
</span>impl </span>GivePatsFuture {
</span>    </span>fn </span>new</span>() -> super </span>Self </span>{ ... } </span>// create in caller's scope
</span>}
</span>
</span>struct </span>GivePatsIntoFuture;
</span>impl </span>IntoFuture </span>for </span>GivePatsIntoFuture {
</span>    </span>type </span>Output = ();
</span>    </span>type </span>IntoFuture = GivePatsFuture;
</span>    </span>fn </span>into_future</span>(</span>self</span>) -> super GivePatsFuture {
</span>        GivePatsFuture::new() </span>// create in caller's scope
</span>    }
</span>}
</span></code></pre>
I probably should have said this sooner, but I'll say it now: in this post I'm
intentionally not bothering with backwards-compat. The point, again, is to break
the complicated design space of "immovable types" into smaller problems we can
tackle one-by-one. Figuring out how to bridge Pin</code> and !Move</code> is something we
will want to figure out at some point - but not now.</p>
As far as async {}</code> and Future</code> are concerned: this should work!
This allows us to freely move around async blocks which desugar into
IntoFuture</code>. And only once we're ready to start polling them do we call
into_future</code> to obtain an impl Future + !Move</code>. A system like that is
equivalent to the existing Pin</code> system, but does not need Pin</code> in its
signature. For good measure, here's how we would be able to rewrite the
signature of Future</code> with this change:</p>
// The current `Future` trait
</span>// using `Pin<&mut Self>`
</span>pub trait </span>Future {
</span>    </span>type </span>Output;
</span>    </span>fn </span>poll</span>(</span>self</span>: Pin<&</span>mut Self</span>>, </span>cx</span>: &</span>mut </span>Context<'_>) -> Poll<</span>Self::</span>Output>;
</span>}
</span>
</span>// The `Future` trait leveraging `Move`
</span>// using `&mut self`
</span>pub trait </span>Future {
</span>    </span>type </span>Output;
</span>    </span>fn </span>poll</span>(&</span>mut </span>self</span>, </span>cx</span>: &</span>mut </span>Context<'_>) -> Poll<</span>Self::</span>Output>;
</span>}
</span></code></pre>
That would also mean: no more Pin</code>-projections. No more incompatibilities with
Drop</code>. Because it's an auto-trait that governs language behavior, as long as
the base rules are sound, the interaction with all other parts of Rust would be sound.</p>
Also, most relevant for me probably, this would make it possible to write future
state machines using methods</em> and functions</em>, rather than the current status
quo where we just lump everything into the poll</code> function body. After having
written a ridiculous amount of futures by hand over the past six years, I can't
tell you how much I'd love to be able to do that.</p>
Motivating example, reworked</h2>
Now that we've covered self-referential lifetimes, in-place construction,
understand async {}</code> should return IntoFuture</code>, and have seen !Move</code>, we're
ready to bring these features together to rework our motivating example. This is
what we started with using regular async/.await</code> code:</p>
async </span>fn </span>give_pats</span>() {
</span>    </span>let</span> data = "</span>chashu tuna</span>".</span>to_string</span>();
</span>    </span>let</span> name = data.</span>split</span>(' ').</span>next</span>().</span>unwrap</span>();
</span>    </span>pat_cat</span>(&name).await;
</span>    println!("</span>patted </span>{name}</span>");
</span>} 
</span>
</span>async </span>fn </span>main</span>() {
</span>    </span>give_pats</span>().await;
</span>}
</span></code></pre>
And here is what that, with these new capabilities, here is what async fn give_pats</code> would be able to desugar to.  Note the 'self</code> lifetime, the !Move</code>
impl for the future, the omission of Pin</code> everywhere, and the in-place
construction of the type.</p>
enum </span>GivePatsState {
</span>    Created,
</span>    Suspend1,
</span>    Complete,
</span>}
</span>
</span>struct </span>GivePatsFuture {
</span>    </span>resume_from</span>: GivePatsState,
</span>    </span>data</span>: Option<String>,
</span>    </span>name</span>: Option<&</span>'self str</span>>,        </span>// ← Note the `'self` lifetime
</span>}
</span>impl </span>!Move for GivePatsFuture {}     </span>// ← This type is immovable
</span>impl </span>Future </span>for </span>GivePatsFuture {
</span>    </span>type </span>Output = ();
</span>    </span>fn </span>poll</span>(&</span>mut </span>self</span>, </span>cx</span>: &</span>mut </span>Context<'_>)  </span>// ← No `Pin` needed
</span>        -> Poll<</span>Self::</span>Output> { ... }
</span>}
</span>
</span>struct </span>IntoGivePatsFuture {}
</span>impl </span>IntoFuture </span>for </span>IntoGivePatsFuture {
</span>    </span>type </span>Output = ();
</span>    </span>type </span>IntoFuture: GivePatsFuture
</span>    </span>fn </span>into_future</span>(</span>self</span>)
</span>        -> super GivePatsFuture {  </span>// ← Writes to a stable addr
</span>        </span>Self </span>{
</span>            resume_from: GivePatsState::Created,
</span>            data: None,
</span>            name: None,
</span>        }
</span>    }
</span>}
</span></code></pre>
And finally, we can then desugar the give_pats().await</code> call to concrete types
we construct and call to completion:</p>
let</span> into_future = IntoGivePatsFuture {};
</span>let mut</span> future = into_future.</span>into_future</span>(); </span>// ← Immovable without `Pin`
</span>loop </span>{
</span>    </span>match</span> future.</span>poll</span>(&</span>mut</span> current_context) {
</span>        Poll::Ready(ready) => </span>break</span> ready,
</span>        Poll::Pending => </span>yield</span> Poll::Pending,
</span>    }
</span>}
</span></code></pre>
And with that, we should have a working example of async {}</code> blocks, desugared
to concrete types and traits that don't use Pin</code> anywhere at all. Accessing
fields within it wouldn't go through any kind of pin projection, and there would
no longer be any need for things like stack-pinning. Immovability would just be
a property of the types themselves, constructed when we need them in the place
where we want to use them.</p>
Oh and I guess just to mention it: functions working with these traits would
always want to use T: IntoFuture</code> rather than T: Future</code>. That's not a big
change, and actually something people should already be doing today. But I
figured I'd mention it in case people are confused about what the bounds should
be for concurrency operations.</p>
Phased initialization</h2>
We didn't show this in our example, but there is one more aspect to
self-referential types worth covering: phased initialization. This is when you
initialize parts of a type at separate points in time. In our motivating example
we didn't have to use that, because the self-references lived inside of an
Option</code>. That means that when we initialized the type we could just pass
None</code>, and things were fine. However, say we did want to initialize a
self-reference, how would we go about that?</p>
struct </span>Cat {
</span>    </span>data</span>: String,
</span>    </span>name</span>: &</span>'self str</span>,
</span>}
</span>impl </span>!Move for Cat {}
</span>
</span>impl </span>Cat {
</span>    </span>fn </span>new</span>(</span>data</span>: String) -> super </span>Self </span>{
</span>        Cat {
</span>            data: "</span>chashu tuna</span>".</span>to_string</span>(),
</span>            name: </span>/* How do we reference `self.data` here? */
</span>        }
</span>    }
</span>}
</span></code></pre>
Now of course because String</code> is heap-allocated, its address is actually stable
and so we could write something like this:</p>
struct </span>Cat {
</span>    </span>data</span>: String,
</span>    </span>name</span>: &</span>'self str</span>,
</span>}
</span>impl </span>!Move for Cat {}
</span>
</span>impl </span>Cat {
</span>    </span>fn </span>new</span>(</span>data</span>: String) -> super </span>Self </span>{
</span>        </span>let</span> data = "</span>chashu tuna</span>".</span>to_string</span>();
</span>        Cat {
</span>            name: data.</span>split</span>(' ').</span>next</span>().</span>unwrap</span>(),
</span>            data, 
</span>        }
</span>    }
</span>}
</span></code></pre>
That's clearly cheating, and not what we want people to have to do. But it does
point us at how the solution here should probably work: we first need a stable
address to point to. And once we have that address, we can refer to it. We can't
do that if we have to build the entire thing in a single go. But what if we
could do it in multiple phases</em>? That's what Niko's recent post on borrow
checking and view
types</a> went into. That would allow us to change our example to instead be written like this:</p>
struct </span>Cat {
</span>    </span>data</span>: String,
</span>    </span>name</span>: &</span>'self str</span>,
</span>}
</span>impl </span>!Move for Cat {}
</span>
</span>impl </span>Cat {
</span>    </span>fn </span>new</span>(</span>data</span>: String) -> super </span>Self </span>{
</span>        </span>super let</span> this = Cat { data: "</span>chashu tuna</span>".</span>to_string</span>() }; </span>// ← partial init
</span>        this.name = this.data.</span>split</span>(' ').</span>next</span>().</span>unwrap</span>();         </span>// ← finish init
</span>        this
</span>    }
</span>}
</span></code></pre>
We initialize the owned data in Cat</code> first. And once we have that, we can then
initialize the references to it. These references would be 'self</code>, we sprinkle
in a super let</code> annotation to indicate we're placing this in the caller's
scope, and everything should subsequently check out.</p>
Migrating from Pin to Move</h2>
What we didn't cover in this post is any migration story from the existing
Pin</code>-based APIs to the new Move</code>-based system. If we want to move off of Pin</code>
in favor of Move</code>, the only path plausible way I see is by minting new traits
that don't carry Pin</code> in its signature, and providing bridging impls from the
old traits to the new traits. A basic conversion with an explicit method could
look like this, though blanket impls could also be a possibility:</p>
pub trait </span>NewFuture {
</span>    </span>type </span>Output;
</span>    </span>fn </span>poll</span>(&</span>mut </span>self</span>, ...) { ... }
</span>}
</span>
</span>pub trait </span>Future {
</span>    </span>type </span>Output;
</span>    </span>fn </span>poll</span>(</span>self</span>: Pin<&</span>mut Self</span>>, ...) { ... }
</span>
</span>    </span>/// Convert this future into a `NewFuture`.
</span>    </span>fn </span>into_new_future</span>(</span>self</span>: Pin<&</span>mut Self</span>>) -> NewFutureWrapper<&</span>mut Self</span>> { ... }
</span>}
</span>
</span>/// A wrapper bridging the old future trait to the new future trait.
</span>struct </span>NewFutureWrapper<</span>'a</span>, F: Future>(Pin<&</span>'a mut</span> F>);
</span>impl </span>!Move for NewFutureWrapper {}
</span>impl</span><</span>'a</span>, F> NewFuture </span>for </span>NewFutureWrapper<</span>'a</span>, F> { ... }
</span></code></pre>
I've been repeating this line for at least three years now: but if we want to
fix the problems with Pin</code>, the first step we need to take is to not make the
problem worse. If the stdlib needs to fix the Future</code> trait once, that sucks
but it's fine and we'll find a way to do it. But if we tie Pin</code> up into a
number of other traits, the problems will compound and I'm no longer sure
whether we can rid ourselves of Pin</code>. And that's a problem, because Pin</code> is
broadly disliked and we actively want to get rid of it.</p>
Compatibility with self-referential types is not only relevant for iteration;
it's a generalized property which ends up interacting with nearly every trait,
function, and language feature. Move</code> just composes with any other trait, and
so there's no need for a special PinnedRead</code> or anything. A type would instead
just implement Read + Move</code>, and that would be enough for a self-referential
reader to function. And we can repeat that for any other combination of traits.</p>
In-place construction of course does change the signature of traits. But in
order to support that in a backwards-compatible way, all we'd need to do is
enable traits to opt-in to "can perform in-place construction"</em>. And being able
to gradually roll-out capabilities like that is exactly why we're working on
effect generics.</p>
pub trait </span>IntoFuture {
</span>    </span>type </span>Output;
</span>    </span>type </span>IntoFuture: Future<Output = </span>Self::</span>Output>;
</span>
</span>    </span>// Marked as compatible with in-place construction, with
</span>    </span>// implementations being able to decide whether they want
</span>    </span>// to use it or not.
</span>    </span>fn </span>into_future</span>(</span>self</span>) -> #[</span>maybe</span>(super)] </span>Self</span>::IntoFuture;
</span>}
</span></code></pre>
If we want self-referential types to be generally useful, they need to
practically compose with most other features we have. And so really, the first
step to getting there is stop stabilizing any new traits in the stdlib which use
Pin</code> in its signature.</p>
Making immovable types movable</h2>
So far we’ve been talking a lot about self-referential types and how we need to
make sure they cannot be moved, because moving them would be bad. But what if we
did allow them to be moved? In C++ this is possible using a feature called
"move constructors"</em>, and if we supported self-referential types in Rust, it
doesn't seem like a big leap to support that too.</p>
Before we go any further I want to preface this: I've heard from people who have
worked with move constructors in C++ that they can be rather tricky to work
with. I haven't worked with them, so I can't speak from experience. Personally I
don't really have any uses where I feel like I would have wanted move
constructors, so I'm not particularly in favor or against supporting them. I'm
writing this section mostly out of academic interest, because I know there will
be people wondering about this. And the rules for how this should work seem
fairly straightforward.</p>
Niko Matsakis recently wrote a two-parter on the Claim</code> trait
(first</a>,
second</a>),
proposing a new Claim</code> trait to fill the gap between Clone</code> and Copy</code>. This
trait would be for types which are "cheap" to clone, such as the Arc</code> and Rc</code>
types. And using
autoclaim</a>,
the compiler would automatically insert calls to .claim</code> as needed. For example
when a move ||</code> closure captures a type implementing Claim</code> but it is already
in use somewhere else - it would automatically call .claim</code> so it would compile.</p>
Enabling immovable types to relocate would work much the same as auto-claiming
would. We would need to introduce a new trait, which we'll call Relocate</code> here,
with a method relocate</code>. Whenever we tried to move an otherwise immovable
value, we would automatically call .relocate</code> instead. The signature of the
Relocate</code> trait would take self</code> as a mutable reference. And return an
instance of Self</code>, constructed in-place:</p>
trait </span>Relocate {
</span>    </span>fn </span>relocate</span>(&</span>mut </span>self</span>) -> super </span>Self</span>;
</span>}
</span></code></pre>
Note the signature of self</code> here: we take it by mutable reference - not owned
nor shared. That is because what we're writing is effectively the immovable
equivalent to Into</code>, but we can't take self by-value - so we have to take it
by-reference instead and tell people to just mem::swap</code> away. Applying this our
earlier Cat</code> example, we would be able to implement it as follows:</p>
struct </span>Cat {
</span>    </span>data</span>: String,
</span>    </span>name</span>: &</span>'self str</span>,
</span>}
</span>impl </span>Cat {
</span>    </span>fn </span>new</span>(</span>data</span>: String) -> super </span>Self </span>{ ... }
</span>}
</span>impl </span>Relocate </span>for </span>Cat {
</span>    </span>fn </span>relocate</span>(&</span>mut </span>self</span>) -> super </span>Self </span>{
</span>        </span>let mut</span> data = String::new(); </span>// dummy type, does not allocate
</span>        mem::swap(&</span>mut </span>self</span>.data, &</span>mut</span> data); </span>// take owned data
</span>        </span>super let</span> cat = Cat { data }; </span>// construct new instance
</span>        cat.name = cat.data.</span>split</span>(' ').</span>next</span>().</span>unwrap</span>(); </span>// create self-ref
</span>        cat
</span>    }
</span>}
</span></code></pre>
We're making one sketchy assumption here: we need to be able to take the owned
data from self</code>, without running into the issue where the data can't be moved
because it is already borrowed from self</code>. This is a general problem we need to
solve, and one way we could for example work around this is by creating dummy
pointers in the main struct to ensure the types are always valid - but we
invalidate the types:</p>
struct </span>Cat {
</span>    </span>data</span>: String,
</span>    </span>dummy_data</span>: String, </span>// never initialized with a value
</span>    </span>name</span>: &</span>'self str</span>,
</span>}
</span>
</span>impl </span>Relocate </span>for </span>Cat {
</span>    </span>fn </span>relocate</span>(&</span>mut </span>self</span>) -> super </span>Self </span>{
</span>        </span>self</span>.name = &</span>self</span>.dummy_data; </span>// no more references to `self.data`
</span>        </span>let</span> data = mem::take(&</span>mut </span>self</span>.data); </span>// shorter than `mem::swap`
</span>        </span>super let</span> cat = Cat { data };
</span>        cat.name = cat.data.</span>split</span>(' ').</span>next</span>().</span>unwrap</span>();
</span>        cat
</span>    }
</span>}
</span></code></pre>
In this example the Cat</code> would implement Move</code> even if it has a 'self</code>
lifetime, because we can freely move around. When a type is dropped after having
been passed to Relocate</code>, it should not call its Drop</code> impl. Because
semantically we're not trying to drop the type - all we're doing is updating its
location in memory. Under these rules access to 'self</code> in structs would be
available both if Self: !Move</code> OR Self: Relocate</code>.</p>
I want to again emphasize that I'm not directly advocating here for the
introduction of move constructors to Rust. Personally I'm pretty neutral about
them, and I can be convinced either way. I mainly wanted to have at least once
walked through the  out the way move constructors could work, because it seems
like a good idea to know a gradual path here should be possible. Hopefully that
point is coming across okay here.</p>
Further reading</h2>
The Pin</code> RFC</a> is an
interesting read as it describes the system for immovable types we ended up
going with today. Specifically the
comparison</a>
between Pin</code> and Move</code>, and section on
drawbacks</a> are
interesting to read back up on. Especially when we compare it with the pin
docs</a>,
and see what was not present in the RFC - but later turned out to be major
practical issues (e.g. pin projections, interactions with the rest of the
language).</p>
Tmandry presented an interesting series (blog
1</a>, blog
2</a>,
talk</a>) on async internals.
Specifically he covers how async {}</code> blocks desugar into Future</code>-based state machines.
This post uses that desugaring as its motivating example, so for those keen to
learn more about what happens behind the scenes, this is an excellent resource.
The section on .await</code>
desugaring</a> in
the Rust reference is also a good read, as it captures the status quo in the
compiler.</p>
More recently Miguel Young de la Sota's (mcyoung)
talk</a> and
crate</a> to support C++ move constructors
is interesting to read up on. Something I haven't fully processed yet, but is
interesting, is the
New</code> trait</a> it
exposes. This can be used to construct types in-place on both the stack and
the heap</a>,
which ideally something like the super let</code>/super Type</code> notation could support
too. You can think of C++'s move constructors as further evolution of immovable
types, so it's no surprise that there are lots of shared concepts.</p>
Two years ago I tried formulating a way we could leverage view types for safe
pin projection
(post</a>),
and I came up short in a few regards. In particular I wasn't sure how to deal
with the interactions with #[repr(packed)]</code>, how to make Drop</code> compatible, and
how to mark Unpin</code> as unsafe</code>. There might be a path for the latter, but I'm
not aware of any practical solutions for the first two issues. This post is
basically a sequel to that post, but changing the premise from: "How can we fix
Pin</code>?" to "How can we replace Pin</code>?".</p>
Niko's series on view types is also well worth reading. His first
post</a>
discusses what view types are, how they'd work, and why they're useful. And in one of
his most recent
posts</a>
he discusses how view types fall into a broader "4-part roadmap for the borrow
checker" (aka: "the borrow checker within"). In his last post he directly covers
phased initialization using view types as well, which is one of the features we
discuss in this post in relation to self-referential types.</p>
Finally I'd suggest looking at the ouroboros
crate</a>.
It enables safe, phased initialization for self-referential types on stable Rust
by leveraging macros and closures. The way it works is that fields using owned
data are initialized first. And then the closures are executed to initialize
fields referencing the data. Phased initialization using view types as described
in this post emulates that approach, but enables it directly from the language
through a generally useful feature.</p>
Conclusion</h2>
In this post we've deconstructed "self-referential types" into four constituent parts:</p>

'unsafe</code> (unchecked) and 'self</code> (checked) lifetimes which will make it
possible to express self-referential lifetimes.</li>
A moral equivalent to super let</code> / -> super Type</code> to safely support
out-pointers.</li>
A way to backwards-compatibly add optional -> super Type</code> notations.</li>
A new Move</code> auto-trait which governs access to move operations.</li>
A view types feature which will make it possible to construct
self-referential types without going through an Option</code> dance.</li>
</ol>
The final insight this post provides is that today's Pin</code> + Unpin</code> system can
be emulated with Move</code> by creating Move</code> wrappers which can return !Move</code>
types. In the context of async, the pattern would be to construct an impl IntoFuture + Move</code> wrapper, which constructs an impl Future + !Move</code> future in-place via an out-pointer.</p>
People generally dislike Pin</code>, and as far as I can tell there is broad support
for exploring alternative solutions such as Move</code>. Right now the only trait
that uses Pin</code> in the stdlib is Future</code>. In order to facilitate a migration
off of Pin</code> to something like Move</code>, we would do well not to further introduce
any Pin</code>-based APIs to the stdlib. Migrating off of a single API will take
effort, but seems ultimately doable. Migrating off of a host of APIs will take
more effort, and makes it more likely we'll forever be plagued with the
difficulties of Pin</code>.</p>
The purpose of this post has been to untangle the big scary problem of
"immovable types" into its constituents parts so we can begin tackling them
one-by-one. None of the syntax or semantics in this post are meant to be
concrete or final. I mainly wanted to have at least once walked through
everything required to make immovable types work - so that others can dig in,
think along, and we can begin refining concrete details.</p>

in-place construction seems surprisingly simple?
Sat, 22 Jun 2024 00:00:00 +0000
introduction</h2>
I've been thinking a little bit about self-referential types recently, and one
of its requirements is that a type, when constructed, has a fixed location in
memory 1</a></sup>. That's needed, because a reference is pointer to a
memory address - and if the memory address it points to changes, that would
invalidate the pointer, which can lead to undefined behavior.</p>
^{1</sup>
Eric Holk and I toyed around for a little while with the idea
of offset-schemes, where we only track offsets from the start of structs. But
that gets weird once you involve non-linear memory such as when using references
in the heap. Using real, actual addresses is probably the better solution since
that should work in all cases - even if it brings its own sets of challenges.</p>
</div>}Where this becomes tricky is that returning data from a function constitutes a
move</em>. A constructor which creates a type within itself and returns it, will
change the address of the type it constructs. Take this program which constructs
a type Cat</code> and in the function Cat::new</code> (playground</a>):</p>
use </span>std::ptr::addr_of;
</span>
</span>struct </span>Cat { </span>age</span>: </span>u8 </span>}
</span>impl </span>Cat {
</span>    </span>fn </span>new</span>(</span>age</span>: </span>u8</span>) -> </span>Self </span>{
</span>        </span>let</span> this = </span>Self </span>{ age };
</span>        dbg!(addr_of!(this)); </span>// ← first call to `addr_of!`
</span>        this
</span>    }
</span>}
</span>
</span>fn </span>main</span>() {
</span>    </span>let</span> cat = Cat::new(</span>4</span>);
</span>    dbg!(addr_of!(cat));      </span>// ← second call to `addr_of!`
</span>}
</span></code></pre>
If we run this program we get the following output:</p>
[src/main.rs:7:9] addr_of!(this) = 0x00007ffe0b3575d7 # first call
</span>[src/main.rs:14:5] addr_of!(cat) = 0x00007ffe0b357747 # second call
</span></code></pre>
This shows that the address of Cat</code> within</em> Cat::new</code> is different from the
address once it has been returned from the function. Returning types from
functions means changing addresses. Languages like C++ have ways to work around
this by providing something called move constructors</em>, enable types to update
their own internal addresses when they are moved in memory. But instead, what if
we could just construct types in-place so they didn't have to move in the first
place?</p>
in-place construction</h2>
We already perform in-place construction when desugaring async {}</code> blocks, so
this is something we know how to do. And in the ecosystem there is also the
moveit</a> and
ouroboros</a> crates. These are all great, but they all
do additional things like "self-references" or "move constructors". Constructing in-place rather than returning from a function can be useful just for the sake of reducing copies - so let's roll our own version of just that.</p>
The way we can do this is by creating a stable location in memory we can store
our value in. Rather than returning a value, a constructor should take a mutable
reference to this memory location and write directly into it instead. And
because we're starting with a location</em> and write into it later, this location
needs to be MaybeUninit</code>. If we put those pieces together, we can adapt our
earlier example to the following (playground</a>):</p>
use </span>std::ptr::{addr_of, addr_of_mut};
</span>use </span>std::mem::MaybeUninit;
</span>
</span>struct </span>Cat { </span>age</span>: </span>u8 </span>}
</span>impl </span>Cat {
</span>    </span>fn </span>new</span>(</span>age</span>: </span>u8</span>, </span>slot</span>: &</span>mut </span>MaybeUninit<</span>Self</span>>) {
</span>        </span>let</span> this: </span>*mut Self </span>= slot.</span>as_mut_ptr</span>();
</span>        </span>unsafe </span>{ 
</span>           addr_of_mut!((*this).age).</span>write</span>(age);
</span>           dbg!(addr_of!(*this));   </span>// ← second call to `addr_of!`
</span>        };
</span>    }
</span>}
</span>
</span>fn </span>main</span>() {
</span>    </span>let mut</span> slot = MaybeUninit::uninit();
</span>    dbg!(addr_of!(slot));      </span>// ← first call to `addr_of!`
</span>    Cat::new(</span>4</span>, &</span>mut</span> slot);
</span>    </span>let</span> cat: &</span>mut</span> Cat = </span>unsafe </span>{ (slot).</span>assume_init_mut</span>() };
</span>    dbg!(addr_of!(*cat));      </span>// ← third call to `addr_of!`
</span>}
</span></code></pre>
If we run the program it will print the following:</p>
[src/main.rs:15:5] addr_of!(slot) = 0x00007ffc9daa590f  # first call
</span>[src/main.rs:9:9] addr_of!(*this) = 0x00007ffc9daa590f  # second call
</span>[src/main.rs:18:5] addr_of!(*cat) = 0x00007ffc9daa590f  # third call
</span></code></pre>
To folks who aren't used to writing unsafe Rust this might look a little
overwhelming. But what we've done is a fairly mechanical translation. Rather
than returning the type Self</code>, we're created a MaybeUninit<Self></code> and passed
it by-reference. The constructor then writes into it, initializing the memory.
From that point onward, all references to Cat</code> are valid and can be assumed to
be initialized.</p>
Unfortunately calling assume_init</code> on the actual value</em> of Cat</code> is not
possible because the compiler treats that as a move - which makes sense since it
takes a type and returns another. But that's mostly a limitation of how we're
doing things - not what we're doing.</p>
indirect in-place construction</h2>
Now what happens if there is a degree of indirection? What if rather than
construct just a Cat</code>, we want to construct a Cat</code> inside of a Bed</code>. We would
have to take the memory location of the outer type, and use that as the location
for the inner type. Let's extend our first example by doing exactly that (playground</a>):</p>
use </span>std::ptr::addr_of;
</span>
</span>struct </span>Bed { </span>cat</span>: Cat }
</span>impl </span>Bed {
</span>    </span>fn </span>new</span>() -> </span>Self </span>{
</span>        </span>let</span> cat = Cat::new(</span>4</span>);
</span>        </span>Self </span>{ cat }
</span>    }
</span>}
</span>
</span>struct </span>Cat { </span>age</span>: </span>u8 </span>}
</span>impl </span>Cat {
</span>    </span>fn </span>new</span>(</span>age</span>: </span>u8</span>) -> </span>Self </span>{
</span>        </span>let</span> this = </span>Self </span>{ age };
</span>        dbg!(addr_of!(this)); </span>// ← first call to `addr_of!`
</span>        this
</span>    }
</span>}
</span>
</span>fn </span>main</span>() {
</span>    </span>let</span> bed = Bed::new();
</span>    dbg!(addr_of!(bed));      </span>// ← second call to `addr_of!`
</span>    dbg!(addr_of!(bed.cat));  </span>// ← third call to `addr_of!`
</span>}
</span></code></pre>
If we run the program it will print the following:</p>
[src/main.rs:15:9] addr_of!(this) = 0x00007fff3910702f
</span>[src/main.rs:22:5] addr_of!(bed) = 0x00007fff391071b7
</span>[src/main.rs:23:5] addr_of!(bed.cat) = 0x00007fff391071b7
</span></code></pre>
Adapting our return-based example to preserve referential stability is once
again very mechanical. Rather than returning Self</code> from a function, we pass
a mutable reference to MaybeUninit<Self></code>. In for Cat</code> to be constructed in
Bed</code>, all we have to do is make sure Bed</code> contains a slot for Cat</code> to be
written to. Put together we end up with the following (playground</a>):</p>
use </span>std::ptr::{addr_of, addr_of_mut};
</span>use </span>std::mem::MaybeUninit;
</span>
</span>struct </span>Bed { </span>cat</span>: MaybeUninit<Cat> }
</span>impl </span>Bed {
</span>    </span>fn </span>new</span>(</span>slot</span>: &</span>mut </span>MaybeUninit<</span>Self</span>>) {
</span>        </span>let</span> this: </span>*mut Self </span>= slot.</span>as_mut_ptr</span>();
</span>        Cat::new(</span>4</span>, </span>unsafe </span>{ &</span>mut </span>(*this).cat });
</span>    }
</span>}
</span>
</span>struct </span>Cat { </span>age</span>: </span>u8 </span>}
</span>impl </span>Cat {
</span>    </span>fn </span>new</span>(</span>age</span>: </span>u8</span>, </span>slot</span>: &</span>mut </span>MaybeUninit<</span>Self</span>>) {
</span>        </span>let</span> this: </span>*mut Self </span>= slot.</span>as_mut_ptr</span>();
</span>        </span>unsafe </span>{ 
</span>            addr_of_mut!((*this).age).</span>write</span>(age);
</span>            dbg!(addr_of!(*this)); </span>// ← second call to `addr_of!`
</span>        };
</span>    }
</span>}
</span>
</span>fn </span>main</span>() {
</span>    </span>let mut</span> slot = MaybeUninit::uninit();
</span>    dbg!(addr_of!(slot));      </span>// ← first call to `addr_of!`
</span>    Bed::new(&</span>mut</span> slot);
</span>    </span>let</span> bed: &</span>mut</span> Bed = </span>unsafe </span>{ (slot).</span>assume_init_mut</span>() };
</span>    dbg!(addr_of!(*bed));      </span>// ← third call to `addr_of!`
</span>}
</span></code></pre>
Which if we run the program will print the following addresses. These are all
the same because Cat</code> is the only field inside of Bed</code>, so they happen to
point to the same memory location:</p>
[src/main.rs:23:5] addr_of!(slot) = 0x00007fff8271d86f   # first call
</span>[src/main.rs:17:9] addr_of!(*this) = 0x00007fff8271d86f  # second call
</span>[src/main.rs:26:5] addr_of!(*bed) = 0x00007fff8271d86f   # third call
</span></code></pre>
future possibilities</h2>
If we squint here it's not hard to see how this could be converted into a
language feature. In this post we've mechanically performed a transformation by
hand. Rather than returning a type T</code> from a function, we're taking a &mut MaybeUninit<T></code> and writing into that. It feels like it's basically just a
spicy return</em>; and it seems like something we could introduce some kind of
notation for.</p>
Though admittedly things get trickier once we want to also enable self-references,
perform phased initialization, immovable types, and so on. But those all depend
on being able to write to a fixed place in memory - and it feels like perhaps
these are concepts which we can decouple from one another? Anyway, if we just
take in-place construction as a feature, I think we might be able to get away
with something like this for our first example:</p>
use </span>std::ptr::addr_of;
</span>
</span>struct </span>Cat { </span>age</span>: </span>u8 </span>}
</span>impl </span>Cat {
</span>    </span>fn </span>new</span>(</span>age</span>: </span>u8</span>) -> #[</span>in_place</span>] </span>Self </span>{ </span>// ← new notation
</span>        </span>Self </span>{ age }
</span>    }
</span>}
</span>
</span>fn </span>main</span>() {
</span>    </span>let</span> cat = Cat::new(</span>4</span>);
</span>    </span>// ^ cat was constructed in-place
</span>}
</span></code></pre>
This is obviously not a new idea - but it stood out to me how simple the actual
underpinnings of in-place construction seem. The change from a regular return to
an in-place return feels mechanical in nature - and that seems like a good sign. For good measure let's also adapt our second example:</p>
use </span>std::ptr::addr_of;
</span>
</span>struct </span>Bed { </span>cat</span>: Cat }
</span>impl </span>Bed {
</span>    </span>fn </span>new</span>() -> #[</span>in_place</span>] </span>Self </span>{         </span>// ← new notation
</span>        </span>Self </span>{ cat: Cat::new(</span>4</span>) }
</span>        </span>//     ^ cat was constructed in-place
</span>    }
</span>}
</span>
</span>struct </span>Cat { </span>age</span>: </span>u8 </span>}
</span>impl </span>Cat {
</span>    </span>fn </span>new</span>(</span>age</span>: </span>u8</span>) -> #[</span>in_place</span>] </span>Self </span>{  </span>// ← new notation
</span>        </span>Self </span>{ age }
</span>    }
</span>}
</span>
</span>fn </span>main</span>() {
</span>    </span>let</span> bed = Bed::new();
</span>    </span>// ^ bed was constructed in-place
</span>}
</span></code></pre>
Admittedly I haven't read up on the 10-year discourse of
placement-new</a>, so I assume
there are plenty of details left out that make this hard in the general sense.
Things like heap-addresses and intermediate references. But for the simplest
case? It seems surprisingly doable. Not quite something which we can
proc-macro - but not far off either. And maybe scoping that as a first target
would be enough? I don't know.</p>
The connection to super let</h2>
edit 2024-06-25</strong>: This section was added after first publishing this post.</em></p>
Jack Huey reached out after publishing this post, and
mentioned there might be a connection with the super let</code>
feature</a>. I think that's a super interesting
point, and it's not hard to see why! Take this example from Mara's post:</p>
let</span> writer = {
</span>    println!("</span>opening file...</span>");
</span>    </span>let</span> filename = "</span>hello.txt</span>";
</span>    </span>super let</span> file = File::create(filename).</span>unwrap</span>();
</span>    Writer::new(&file)
</span>};
</span></code></pre>
The super let file</code> notation here allows the file</code>'s lifetime to be scoped to
the outer scope, making it valid for Writer</code> to take a reference to file</code> and
return. Without super let</code> this would result in a lifetime error. Its vibes are
very similar to the #[in_place]</code> notation we posited in this post.</p>
Perhaps there a synthesis of both features could exist to create a form of
"generalized super scope" feature? There definitely appears like there might be
some kind of connection. Like, we could imagine writing something like super let</code> to denote a "value which is allocated in the caller's frame"</em>. And a
function returning super Type</code> to signal from the type signature that this is
actually an out-pointer 2</a></sup>.</p>
^{2</sup>
Shout out to James Munns for teaching me about the term "outpointer" - that's apparently the term C++ uses for this using the outptr</code> keyword.</p>
</div>}use </span>std::ptr::addr_of;
</span>
</span>struct </span>Cat { </span>age</span>: </span>u8 </span>}
</span>impl </span>Cat {
</span>    </span>fn </span>new</span>(</span>age</span>: </span>u8</span>) -> super </span>Self </span>{
</span>        </span>super let</span> this = </span>Self </span>{ age };  </span>// declare in the caller's frame
</span>        dbg!(addr_of!(this));
</span>        this
</span>    }
</span>}
</span></code></pre>
It's worth nothing though that this is not an endorsement for actually going
with super let</code> or super Type</code> - but merely to speculate how there might be a
possible connection between both features. I think it's fun, and the connection
between both seems worthy of further exploration!</p>
conclusion</h2>
In this post I've shown how we can construct types in-place using MaybeUninit</code>
which surprised me how simple it ended up being. I mostly wanted to have gone
through the motions at least once - and now I have, and that was fun!</p>
edit 2024-06-22:</strong> Thanks to Jordan Rose and Simon Sapin for helping un-break
the unsafe pointer code in an earlier version of this post. Goes to show:
Rust's pointer ergonomics really could use an
overhaul</a>.</p>

Context Managers: Undroppable Types for Free
Wed, 05 Jun 2024 00:00:00 +0000
In program language design I'm a big fan of features which fall out of other,
more general features. People occasionally talk about both "unleakable" and
"undroppable" types. I see a lot of value in "unleakable" types because if a
type is "unleakable" we can guarantee the type will always have its destructor
called. This would allow us to use Drop</code> to be used to uphold safety
invariants, which will make it possible to write things like "task scopes". For
type system nerds: yes, this means Rust would be able to uphold linear
invariants via its type system, AKA equip Rust with linear types.</p>
"Undroppable types" have a more limited application. There is some element of
undroppability needed to ensure async destructors can only be called in
async contexts, but that's not the same things as a generalized system for
"undroppable types". The main use for undroppable types is to be able to
guarantee</em> that some type, when destructed, always either takes or returns some
kind of value.</p>
^{1</sup></div>}I believe it should be possible to express "undroppable types" using Tmandry's
design for
context/capabilities</a>.
This design introduces a system similar to Python's "context manager" API.
That systems enables additional arguments to be passed to specific trait impls
(like we do with thread-locals today, but via function signatures), which must
be in-scope when the trait is invoked. Assuming the implementation is general
enough, that would enable us to require Drop</code> impls to take extra arguments like so:</p>
struct </span>Tuna {}
</span>struct </span>Cat { </span>name</span>: &</span>'static str </span>}
</span>impl </span>Drop </span>for </span>Cat {
</span>    </span>fn </span>drop</span>(&</span>mut </span>self</span>) with Tuna { </span>// ← Requires `Tuna` to be in scope when invoked
</span>        println!("</span>{}</span> is snacking on tuna</span>", </span>self</span>.name);
</span>    }
</span>}
</span>
</span>fn </span>main</span>() {
</span>    </span>let</span> cat = Cat { name: "</span>Chashu</span>" };
</span>    with Tuna {} {  </span>// ← Bring `Tuna` into the `with`-scope here
</span>        </span>drop</span>(cat);  </span>// ← Prints: "Chashu is snacking on tuna"
</span>    }
</span>}
</span></code></pre>
That should work pretty well for types which want to require specific arguments
when dropped. Because if Tuna</code> would not be in scope, the type Cat</code>  could not
be dropped. Now where this falls short is if we want to require destructors to
return values too. This is not a MAY return values type of deal, but a MUST
return values. If returning a value was not a hard requirement, then we could
just add some method and use Drop</code> for all other case. But for true
"undroppable types", it must be impossible for a type to be dropped. That means:
the only way to destruct the value should be by calling the method on it. It
should be possible to express this by leveraging private constructors and
contexts like so:</p>
struct </span>Priv { </span>_priv</span>: () } </span>// ← Type can only be constructed in this module
</span>struct </span>Loaf { </span>_priv</span>: () } </span>// ← Type can only be constructed in this module
</span>
</span>struct </span>Cat { </span>name</span>: &</span>'static str </span>}
</span>impl </span>Cat {
</span>    </span>// Consume `Cat` and returns a sleepy little loaf
</span>    </span>fn </span>nap</span>(</span>self</span>) -> Loaf {
</span>        with Priv { _priv: () } { </span>// ← Create a context to drop `Cat`
</span>            </span>drop</span>(</span>self</span>);           </span>// ← Drop the `Cat` instance
</span>            Loaf { _priv: () }    </span>// ← Return some type
</span>        }
</span>    }
</span>}
</span>
</span>/// The type `Priv` cannot be externally constructed, meaning this
</span>/// `Drop` impl prevents the type from being dropped outside of the
</span>/// `Cat::nap` method.
</span>impl </span>Drop </span>for </span>Cat {
</span>    </span>fn </span>drop</span>(&</span>mut </span>self</span>) with Priv {}
</span>}
</span>
</span>fn </span>main</span>() {
</span>    </span>let</span> cat = Cat { name: "</span>Chashu</span>" };
</span>    cat.</span>nap</span>(); </span>// ✅ Compiles as expected.
</span>
</span>    </span>let</span> cat = Cat { name: "</span>Nori</span>" };
</span>    </span>drop</span>(cat); </span>// 💥 Compiler: Cannot invoke `impl Drop for Cat`, `Priv` is not in scope.
</span>}
</span></code></pre>
I think this is pretty fun: a more general feature with wide applicability could
be leveraged to solve a more niche feature with more limited applications. I bet
if custom diagnostics are generalized enough, we could even improve the error
message here, pointing people to call Cat::nap</code> when they can't just drop the
type.</p>
Of course there are a number of other practical applications that come with
undroppable types. The main one being the fact that both any function may
panic, and panic currently equates recoverable control flow. For anyone to even
attempt using undroppable types outside of a demo scenario, they'd have to
grapple with that. But luckily: that again is something that can be resolved
independently from this. I'm happy to be able to show that "undroppable
types" does not need to be its own, standalone feature, but instead can be expressed using a more general system 2</a></sup>.</p>
^2</sup>For anyone wondering to themselves: "If undroppable
types are more restrictive than unleakable types, can't we just use undroppable
types everywhere instead?"</em> - The answer is, "Not practically, no"</em>. Unleakable
types add minor restrictions to the language (e.g. no mem::forget</code>, no Arc</code>,
etc.). Undroppable types add additional major restrictions (e.g. no closure
captures, no holding values across ?</code>). Some limitations are acceptable, too
many limitations make the experience painful. For more on this, read my 2023
post: Linearity and
Control</a>.</p>
</div>

Tasks are the wrong abstraction
Sat, 27 Apr 2024 00:00:00 +0000
Okay, so you've probably read the title and are thinking: "Hey is this
clickbait?"</em> - and no dear reader, it is not. I'm fairly convinced at this point
that tasks, as an abstraction, are not right for Rust and we would do better to
achieve parallel execution through composition. However when it comes to task
scheduling</em> things seem less clear, and I would like to have more data
available.  Because task scheduling strategies determine which APIs we can
provide, and as we're looking to standardize the async Rust ecosystem, this will
end up being important.</p>
So okay, big claim - "Tasks are not the wrong abstraction"</em>. Tasks are a staple
in async Rust (and I should know), and task spawning has been around pretty much
since the beginning. However, two relatively recent developments have made me
start to question whether tasks themselves actually carry their weight, both for
single-threaded concurrent workloads and parallel multi-threaded workloads:</p>

Both Bytedance's (TikTok's) monoio</code>
crate</a> and the glommio</code>
crate</a>
use a thread-per-core architecture and appear to scale significantly better than
Tokio's work-stealing architecture on
networking benchmarks</a>.
This raises questions about tasks as primitives for parallel async
execution.</strong> (edit: I mean "questions" literally here - these are interesting
claims which we should research more closely. We get into that more later on in
this post).</li>
Single-threaded executors introduce 'static</code> lifetimes, often do not
correctly propagate errors and cancellation, and in recent testing have shown to
perform up to 2-3x
worse</a>
than their structured counterparts. We have alternative APIs available for
concurrency which do not have these issues. This raises questions about tasks
as primitives for concurrent async execution.</strong></li>
</ol>
In this post I want to discuss task spawning for just concurrent but also
parallel execution. I want to show some of the issues both of these approaches
run into, show how we can do better, and talk about what we need more data on.
Finally I want to speculate a little about where we could go with async
parallelism and concurrency in Rust. But to save everyone some reading, here's
some code roughly summarizing the first half of this post 1</a></sup>:</p>
^{1</sup>
Note that the third example here says: "structured". What I
actually mean is: as structured as is possible when working within the current
limitations of the language. But this API would be entirely structured if we
didn't have those limitations.</p>
</div>}// concurrent execution of two futures today (structured)
</span>let</span> a = async { </span>1 </span>};              </span>// ← does nothing until .awaited
</span>let</span> b = async { </span>2 </span>};              </span>// ← does nothing until .awaited
</span>let </span>(a, b) = (a, b).</span>join</span>().await; </span>// ← concurrent `.await`
</span>
</span>// parallel execution of two futures today (unstructured)
</span>let</span> a = async_std::spawn(async { </span>1 </span>});  </span>// ← begins parallel execution when created
</span>let</span> b = async_std::spawn(async { </span>2 </span>});  </span>// ← begins parallel execution when created
</span>let</span> a = a.await;                        </span>// ← await order does not affect execution
</span>let</span> b = b.await;                        </span>// ← await order does not affect execution
</span>
</span>// parallel execution of two futures as proposed (structured)
</span>let</span> a = async { </span>1 </span>}.</span>par</span>();         </span>// ← does nothing until .awaited
</span>let</span> b = async { </span>2 </span>}.</span>par</span>();         </span>// ← does nothing until .awaited
</span>let </span>(a, b) = (a, b).</span>join</span>().await;  </span>// ← parallel `.await`
</span></code></pre>
Preface: concurrent and parallel execution</h2>
Note (2024-04-27): this section was added after repeated questions about the
difference between parallelism and concurrency.</em></p>
In order to make sense of this post, it's important to understand the
differences between parallelism and concurrency, as well as parallel execution
and concurrent execution. These are related but distinct terms, and it can take
some time to internalize. My favorite definition of the differences between
these comes from Aaron Turon's PhD
thesis</a>:</p>

Concurrency is a system-structuring mechanism, parallelism is a resource.</p>
</blockquote>
Put concretely: "concurrency" refers to the way we schedule work. While
"parallelism" refers to e.g. the amount of cores a computer has. If we want to
perform parallel execution of work</em>, we have to schedule work concurrently
using the system's resources for parallelism. We can plot the relationship of
parallelism and concurrency in a 2x2 table:</p>
</th> no parallelism</th> has parallelism</th></tr></thead>

sequential scheduling</strong></td> sequential execution</td> sequential execution</td></tr>
concurrent scheduling</strong></td> concurrent execution</td> parallel execution</td></tr>
</tbody></table>
This table is probably going to surprise some folks. What we're seeing here is
that even if we use multiple threads, it's still possible to achieve
sequential</em> execution. How can that be? Well dear reader, imagine some
exclusive resource shared between N threads. In order for any thread to make
progress they must take an exclusive lock out on that resource. That would
certainly make use of multiple threads; but execution would be entirely
sequential - only one thread can make progress at any given moment. In order to
achieve parallel execution</em> it's not enough to just make use of parallelism</em>;
we also have to schedule concurrently.</p>
Conversely it's also possible to schedule work concurrently despite not having
access to any parallelism. An example of this is for example: race two timers
with each other. We're waiting on both at the same time, despite not having
multiple threads. This is an example of concurrent execution</em> without any
access to resources for parallelism.</p>
Tasks are the wrong abstraction for parallel async execution</h2>
Cq. 2019 I helped popularize the reasoning that: "Tasks should function like
the async/.await</code> equivalent of threads"</em>. Among other things that meant that
in the runtime</code> crate and subsequently async-std</code> we made sure that tasks
always returned JoinHandle</code>s, and that those handles could be awaited to obtain
values. Prior to that it was common to manually create async one-shot channels
to obtain values from tasks
(src</a> 2</a></sup>):</p>
^{2</sup>
Romio and Juliex were the reactor and executor Boats and I worked on
between 2018-2019. It's a pretty good snapshot of what the state of the art of
APIs looked like back then.</p>
</div>}//! What using tasks was like cq. December 2018
</span>
</span>use </span>std::thread;
</span>use </span>futures::{executor, channel::oneshot};
</span>
</span>fn </span>main</span>() {
</span>    </span>let mut</span> handles = vec![];
</span>
</span>    </span>for </span>_ in </span>0</span>..</span>10 </span>{
</span>        </span>let </span>(sender, receiver) = oneshot::channel();
</span>        handles.</span>push</span>(receiver);
</span>        juliex::spawn(async </span>move </span>{
</span>            </span>let</span> id = thread::current().</span>id</span>();
</span>            sender.</span>send</span>(id);
</span>        })
</span>    }
</span>
</span>    </span>for</span> handler in handles {
</span>        </span>let</span> id = handler.await; </span>// this was actually `await!` then.
</span>        println!("</span>handler returned from thread: </span>{id:?}</span>");
</span>    }
</span>}
</span></code></pre>
That rationale that "tasks are like async threads"</em> has stuck around, and I
think it is wrong. See, concurrency and parallelism in async Rust are different
than in non-async Rust. The Thread</code> abstraction packages both concurrency</em> and
parallelism</em> into a single abstraction. Whereas in async Rust the two can be
decoupled: we can execute any number of futures concurrently, and we don't need
to also make us of parallelism for it. Let's walk through some examples,
starting with parallel execution using unstructured thread handles:</p>
use </span>std::thread;
</span>
</span>let mut</span> handles = vec![];
</span>handles.</span>push</span>(thread::spawn(|| {
</span>    </span>1 </span>// ← the result of some computation
</span>}));
</span>handles.</span>push</span>(thread::spawn(|| {
</span>    </span>2 </span>// ← the result of another computation
</span>}));
</span>
</span>let</span> output = handles.</span>into_iter</span>().</span>map</span>(|</span>h</span>| h.</span>join</span>().</span>unwrap</span>()).</span>sum</span>();
</span>assert_eq!(output, </span>3</span>);
</span></code></pre>
Rust does not provide us with a way to say that no, we don't actually want to
leverage parallelism here - we just want concurrency. That's why thread::spawn</code>
always takes a + Send</code> bound on the closure. In async Rust however, we can just
choose to execute work concurrently via the Join</code> family of APIs. Here's an
example using futures-concurrency</code>:</p>
use </span>futures_concurrency::prelude::*;
</span>
</span>let mut</span> handles = vec![];
</span>handles.</span>push</span>(async {
</span>    </span>1 </span>// ← the result of some computation
</span>}));
</span>handles.</span>push</span>(async {
</span>    </span>2 </span>// ← the result of another computation
</span>}));
</span>
</span>let</span> output = handles.</span>join</span>().await.</span>into_iter</span>().</span>sum</span>();
</span>assert_eq!(output, </span>3</span>);
</span></code></pre>
Structurally this is very similar to the unstructured threads example; however
because futures are lazy and automatically propagate cancellation, they can be
considered structured 3</a></sup>. Though typically we'd probably write this
example like this instead:</p>
^{3</sup>
I've gone into details about the limitations of structured concurrency in Rust today in other posts. TLDR: we need an async version of Drop</code> to actually solve all cases.</p>
</div>}use </span>futures_concurrency::prelude::*;
</span>
</span>let</span> a = async { </span>1 </span>};
</span>let</span> b = async { </span>2 </span>};
</span>
</span>let</span> output = (a, b).</span>join</span>().await.</span>into_iter</span>().</span>sum</span>(); </span>// ← executes futures concurrently
</span>assert_eq!(output, </span>3</span>);
</span></code></pre>
Now what about parallelism? Well, the point I'm trying to make is that we can
achieve parallel execution through composition</em> rather than defining new APIs.
It's common practice today to resort to task::spawn</code> APIs for this, mirroring
the thread::spawn</code> APIs:</p>
use </span>async_std::task;
</span>
</span>let mut</span> handles = vec![];
</span>handles.</span>push</span>(task::spawn(async {
</span>    </span>1 </span>// ← the result of some computation
</span>}));
</span>handles.</span>push</span>(task::spawn(async {
</span>    </span>2 </span>// ← the result of another computation
</span>}));
</span>
</span>let</span> output = handles.</span>into_iter</span>().</span>map</span>(|</span>h</span>| h.await).</span>sum</span>();
</span>assert_eq!(output, </span>3</span>);
</span></code></pre>
There's a pretty noticeable difference between the previous two examples: one
family of async APIs for concurrency, and another family of APIs for both
concurrency and parallelism. My pitch here is different: I believe the right way
to achieve parallel execution is through composition</em>. What we need is not
another way to schedule async work; what we need is a way to define a
parallelizable future</em>. And that's something I've prototyped in my tasky</code>
*crate</a>:</p>
use </span>futures_concurrency::prelude::*;
</span>use </span>tasky::prelude::*;
</span>
</span>let</span> a = async { </span>1 </span>}.</span>par</span>(); </span>// ← added `.par` to create a `ParallelFuture`
</span>let</span> b = async { </span>2 </span>}.</span>par</span>(); </span>// ← added `.par` to create a `ParallelFuture`
</span>
</span>let</span> output = (a, b).</span>join</span>().await.</span>into_iter</span>().</span>sum</span>(); </span>// ← executes two futures in parallel
</span>assert_eq!(output, </span>3</span>);
</span></code></pre>
This approach makes it so we have one way of scheduling concurrent execution,
and resources themselves are responsible for deciding whether they should be
parallelizable or not. Again: async Rust allows us to decouple parallelism
from concurrency in ways not possible in non-async Rust; and so we should design
our APIs in ways which leverage that decoupling.</strong></p>
With ParallelizableFuture</code> work doesn't start until it is first .await</code>ed.
This makes it behave just like any other future. Unlike task handles you can't
just fire and forget it; you have to be actively .await</code>ing it to make forward
progress. That means a value is always returned, and cancellation will always
propagate. And once we have async destructors, those should be able to naturally
propagate through the .await</code> points too. This is an API which should be
familiar to use, but hard to misuse. It's setting people up for success when it
comes to things like propagating cancellation and learning about async
concurrency.</p>
Tasks are the wrong abstraction for concurrent async execution</h2>
This point is probably easier to argue than the previous one: using spawn</code> APIs
for just concurrency without also leveraging parallelism is generally not a
great experience. Consider the following example, using task::spawn_local</code>:</p>
use </span>async_std::task;
</span>
</span>let mut</span> handles = vec![];
</span>handles.</span>push</span>(task::spawn_local(async {
</span>    </span>1 </span>// ← the result of some computation
</span>}));
</span>handles.</span>push</span>(task::spawn_local(async {
</span>    </span>2 </span>// ← the result of another computation
</span>}));
</span>
</span>let</span> output = handles.</span>into_iter</span>().</span>map</span>(|</span>h</span>| h.await).</span>sum</span>();
</span>assert_eq!(output, </span>3</span>);
</span></code></pre>
This now does the exact same thing as our earlier Join</code> example, except it
needs to allocate space on the heap to store each individual future. That's a
flat performance tax each task needs to pay; and in this case we've already
shown it's avoidable in ever scenario. But that's just performance; there are
additional restrictions with regards to ergonomics. The signature of
spawn_local</code> is as follows:</p>
pub fn </span>spawn_local</span><F, T>(</span>future</span>: F) -> JoinHandle<T>
</span>where
</span>    F: Future<Output = T> + </span>'static</span>,
</span>    T: </span>'static</span>, 
</span></code></pre>
The 'static</code> lifetime ensures that the future cannot contain any borrows, and
resolving it takes a lot of effort because it isn't natural to the language. An
example of this is the moro</code> crate, which provides an API for "scoped
single-threaded tasks" via an async_scope!</code> macro. The macro is necessary
because the lifetimes required for this currently can't be expressed in the
language. Here is an adaptation of the thread::scope</code> example</a> converted to use moro</code>:</p>
let mut</span> container = vec![</span>1</span>, </span>2</span>, </span>3</span>];
</span>let mut</span> num = </span>0</span>;
</span>
</span>moro::async_scope!(|</span>s</span>| {
</span>    s.</span>spawn</span>(async {
</span>        println!("</span>hello from the first scoped thread</span>");
</span>        dbg!(&container);
</span>    });
</span>    s.</span>spawn</span>(async {
</span>        println!("</span>hello from the second scoped thread</span>");
</span>        num += container[</span>0</span>] + container[</span>2</span>];
</span>    });
</span>    println!("</span>hello from the main thread</span>");
</span>}).await;
</span>
</span>container.</span>push</span>(</span>4</span>);
</span>assert_eq!(num, container.</span>len</span>());
</span></code></pre>
Let's rewrite this using the futures-concurrency</code>, which doesn't rely on tasks,
doesn't enforce 'static</code> lifetimes, and so in turn can freely express what is
being expressed here:</p>
use </span>futures_concurrency::prelude::*;
</span>
</span>let mut</span> container = vec![</span>1</span>, </span>2</span>, </span>3</span>];
</span>let mut</span> num = </span>0</span>;
</span>
</span>let</span> a = async {
</span>    println!("</span>hello from the first future</span>");
</span>    dbg!(&container);
</span>};
</span>
</span>let</span> b = async {
</span>    println!("</span>hello from the second future</span>");
</span>    num += container[</span>0</span>] + container[</span>2</span>];
</span>};
</span>
</span>println!("</span>hello from the main future</span>");
</span>let </span>_ = (a, b).</span>join</span>().await;
</span>container.</span>push</span>(</span>4</span>);
</span>assert_eq!(num, container.</span>len</span>());
</span></code></pre>
There are more complex cases possible where we have dynamically updating sets of
futures or streams we want to append to, which we want to manage as a group.
Getting into that here would mean we'd run long, but for an example of the
problem I'd like to point to Niko's mini-redis
post</a>,
and for an example of how to solve this without tasks or select!</code>, see the
second example of the StreamGroup</code> type</a>.</p>
I realize that by this point we've veered pretty far off the original point of
this section. But it's pretty trivial it probably bears repeating: Tasks</code>
with their 'static</code> bounds and performance overhead seem like a poor fit when
used solely for concurrency.</strong> And while crates like moro</code> can help overcome
some of those challenges, they don't do it fully and don't appear to provide
additional expressivity.</p>
Stealing and sending</h2>
Baked into Rust's async design is the assumption that work-stealing schedulers
represent the pinnacle of performance, and therefor it for example makes sense
that Waker</code> must always be Send</code>. Work-stealing allows threads to "steal" work
from other threads when they're idle. In case one thread has a lot of work
scheduled, and another thread is free, this is supposed to enable lower
latencies and more throughput.</p>
This not without downsides though: in order to facilitate this, it requires that
every future contained within a task is Send</code>. The premise of work-stealing
is that the performance gains it provides are more than the performance
penalties we incur from requiring all futures are Send</code>.</strong> Because making
futures Send</code> not only carries a degree of complexity for the language, it also
comes with inherent performance penalties because it requires synchronization.
You know how you can't use Rc</code> with async/.await</code> - that's a direct artifact
of work-stealing designs.</p>
The Glommio and Monoio runtimes put this premise into question. Neither of them
provide a work-stealing runtime, preferring to use a "thread-per-core" design
instead. But by doing this, they do not require to use additional
synchronization primitives, and seem to perform
better</a>
on networked benchmarks than work-stealing runtimes. Monoio claims 2x the
throughput of Tokio with 4 cores, and 3x the throughput of Tokio with 16
cores.</strong> This is possible because of their thread-per-core design, but likely
also usage of io_uring</code>. I believe we should get updated numbers on this, at least comparing Monoio to the tokio-uring</code> project.</p>
Sharded send bounds</h2>
Tmandry raised an interesting idea a while back: using Rc</code> and other !Send</code>
types inside of Send</code> futures should fine, as long as we can guarantee that all
references are moved together as a group. Over in Swift conversations have
recently begun about a feature called Region Based
Isolation</a>,
describing a very similar idea based on the ideas from: "A Flexible Type System
for Fearless
Concurrency"</a> from
PLDI 2022. The Swift SE describes it as follows:</p>

Through the usage of isolation regions, the language can prove that transferring a non-Sendable value over an isolation boundary cannot result in races because the value (and any other value that might reference it) is not used in the caller after the point of transfer.</p>
</blockquote>
Translating this to Rust, I believe it would allow us to do the following:</p>
let</span> rc = Rc::new(</span>12</span>usize</span>); </span>// ← `!Send` type
</span>task::spawn(async </span>move </span>{   </span>// ← crossing a `Send` boundary
</span>    dbg!(rc);              </span>// ← all references moved: compiler says OK
</span>}).await;
</span></code></pre>
It makes sense that this is all fine: all references to Rc</code> are moved to a new
thread, so that's not an issue. But I don't know whether this can hold for all
!Send</code> types. I don't think it would be, because occasionally threads will be
"special" and so moving a !Send</code> type to another thread even with all
references might end up with trouble. This likely would require an additional
modifier to Send</code>, integrated through the libraries and possibly language. It's
an interesting idea that needs more research before we can consider it viable,
but I wanted to make sure to mention it because it does hold promise.</p>
parallel futures and send bounds</h2>
Both Monoio and Glommio provide a local executor as part of their API. Earlier
on in this post I've explained why "local executors" are not a great
abstraction. That's why for
wasi-async-runtime</code></a>,
which also features a single-threaded runtime, I've chosen not to provide a
spawn</code> API at all. Instead people are encouraged to use libraries such as
futures-concurrency</code> instead.</p>
However, even if work-stealing might not carry its weight in Rust right now, I
think we might still be able to provide a spawn</code> API. That could potentially
even make thread-per-core architectures nicer to express in certain cases where
we want an async fn main</code>. The choice of whether we believe work-stealing
carries its weight will end up deciding what the right API will be to go with.</strong></p>
The task::spawn</code> API for both Tokio and async-std is quite similar and looks
something like this:</p>
pub fn </span>spawn</span><F, T>(</span>future</span>: F) -> JoinHandle<T>
</span>where
</span>    F: Future<Output = T> + Send + </span>'static</span>,
</span>    T: Send + </span>'static</span>, 
</span></code></pre>
It takes a Future</code>, and that Future</code> must be Send</code>. Easy, right? Now what if we compare that with the signature of thread::spawn</code>, which looks like this:</p>
pub fn </span>spawn</span><F, T>(</span>f</span>: F) -> JoinHandle<T>
</span>where
</span>    F: FnOnce() -> T + Send + </span>'static</span>,
</span>    T: Send + </span>'static</span>,
</span></code></pre>
Here we don't have a future, but instead we have a closure. The closure itself
is Send</code>, and the value T</code> is Send</code>. At a glance this might look equivalent
to the task::spawn</code> APIs, but it's not. This becomes clearer if we encode the
original API not as taking Future</code>, but instead as taking IntoFuture</code>:</p>
pub fn </span>spawn</span><F, T>(</span>into_future</span>: F) -> JoinHandle<T>
</span>where
</span>    F: IntoFuture<Output = T> + Send + </span>'static</span>,
</span>    <F as IntoFuture>:IntoFuture: Send + </span>'static</span>,
</span>    T: Send + </span>'static</span>, 
</span></code></pre>
Not only is the thing we pass into the other thread Send</code>; because we want to
allow the ongoing computation itself to be movable between threads the future
itself must also be Send</code>. thread::spawn</code> can use !Send</code> types freely because
once the computation has started, it will not be moved. If we wanted to encode
that, we could do that by changing it to the following signature:</p>
pub fn </span>spawn</span><F, T>(</span>into_future</span>: F) -> JoinHandle<T>
</span>where
</span>    F: IntoFuture<Output = T> + Send + </span>'static</span>,
</span>    T: Send + </span>'static</span>, 
</span>    </span>// `F:IntoFuture` is no longer required to be `Send` here
</span></code></pre>
Whether task::spawn</code> should take Send</code> bounds is a decision we're going to
have to make if we want to encode a standardized spawn API in the stdlib. It's
not necessarily zero-sum though; we could probably encode both. But so far
evidence seems to indicate that if we want maximum performance thread-per-core
is a better approach; while if we want maximum convenience enabling !Send</code>
types to work inside of parallelizable futures may actually be simpler to use.
More data here would certainly be helpful.</p>
A vision for concurrency and parallelism in Rust</h2>
So far I've asked a number of questions, and probably observed a little too
much. I imagine the average async Rust user will be somewhere between confusion
and morbid curiosity of where I'm going with all of this. I want to put a rough
sketch forward of where I, Yosh, would like concurrency and parallelism in Rust
to eventually get to. I believe we could get pretty far if we made concurrency
and parallelism first-class concepts in Rust via two new keywords, which we'll
call par</code> and co</code>.</p>
Take the venerable Rayon ParallelIterator</code> trait. It allows us to iterate over
items in parallel rather than in sequence. While it works great using combinator
APIs; it does not allow us to use for</code>-loops the way we'd expect. What if we
could do that by introducing for par..in</code> loops:</p>
fn </span>square</span>(</span>input</span>: impl Iterator<Item = </span>i32</span>>) -> impl Iterator<Item = </span>i32</span>> {
</span>    gen </span>move </span>{                 </span>// ← current unstable `gen` block notation
</span>        </span>for</span> par num in input { </span>// ← parallel iteration syntax (hypothetical)
</span>            </span>yield</span> num * num;
</span>        }
</span>    }
</span>}
</span></code></pre>
The body of the loop here would be roughly equivalent to Rayon's
ParallelIterator::for_each</code> - with the exception that it doesn't just loop but
returns an iterator. In async Rust we don't yet have async iteration, but we're
currently looking at something like this:</p>
fn </span>square</span>(</span>input</span>: impl async Iterator<Item = </span>i32</span>>) -> impl async Iterator<Item = </span>i32</span>> {
</span>    async gen </span>move </span>{              </span>// ← current unstable `async gen` block notation
</span>        </span>for</span> await num in input {  </span>// ← sequential async iteration syntax (likely)
</span>            </span>yield</span> num * num;
</span>        }
</span>    }
</span>}
</span></code></pre>
This is all unstable and unconfirmed, but it seems likely things will end up
along these lines. The least certain part is the exact shapes and names of
traits, but that's not the important bit here. Now as we've said we can decouple
concurrency and parallelism in async Rust. So what would a concurrent version of
this loop look like? Well, one of the main benefits of async Rust is that we can
run things concurrently - so what if we made that a first-class part of the
language? That's what a hypothetical co</code> keyword could be for:</p>
fn </span>square</span>(</span>input</span>: impl async Iterator<Item = </span>i32</span>>) -> impl async Iterator<Item = </span>i32</span>> {
</span>    async gen </span>move </span>{
</span>        </span>for</span> co await num in input {  </span>// ← concurrent async iteration syntax (hypothetical)
</span>            </span>yield</span> num * num;
</span>        }
</span>    }
</span>}
</span></code></pre>
Whether we'd spell this co.await</code>, co_await</code> or co await</code> doesn't
particularly matter. Making concurrency easier to leverage seems like a nice
thing. In terms of implementation we could leverage my recent work on
ConcurrentStream</code></a> for this. If
we then wanted to extend this to parallelism too, we could instead use par await</code>:</p>
fn </span>square</span>(</span>input</span>: impl async Iterator<Item = </span>i32</span>>) -> impl async Iterator<Item = </span>i32</span>> {
</span>    async gen </span>move </span>{
</span>        </span>for</span> par await num in input {  </span>// ← parallel async iteration syntax (hypothetical)
</span>            </span>yield</span> num * num;
</span>        }
</span>    }
</span>}
</span></code></pre>
This doesn't just need to stop at iterators either; we could integrate this into
futures and async/.await</code> too. Not too different from how Swift's async let</code>
notation works. Remember our earlier example using futures-concurrency</code> to
evaluate futures concurrently?</p>
use </span>futures_concurrency::prelude::*;
</span>
</span>let</span> a = async { </span>1 </span>};
</span>let</span> b = async { </span>2 </span>};
</span>
</span>let</span> output = (a, b).</span>join</span>().await.</span>into_iter</span>().</span>sum</span>(); </span>// ← executes futures concurrently
</span>assert_eq!(output, </span>3</span>);
</span></code></pre>
It would be nice if the compiler could perform control-flow analysis directly,
and for automatically schedule concurrent execution of futures where permitted, as long as they were called with .co.await</code>:</p>
let</span> a = async { </span>1 </span>}.co.await; </span>// ← concurrent await syntax (hypothetical)
</span>let</span> b = async { </span>2 </span>}.co.await; </span>// ← concurrent await syntax (hypothetical)
</span>assert_eq!(a + b, </span>3</span>);
</span></code></pre>
And what about parallelism? Well, we should also be able to compose .par</code> with
.await</code> to achieve parallel async execution:</p>
let</span> a = async { </span>1 </span>}.par.await; </span>// ← parallel await syntax (hypothetical)
</span>let</span> b = async { </span>2 </span>}.par.await; </span>// ← parallel await syntax (hypothetical)
</span>assert_eq!(a + b, </span>3</span>);
</span></code></pre>
One of the main appeals of representing async operations as types is that we can
then arbitrarily combine them with other futures to achieve concurrent
execution. Neither future here needs to be 'static</code> to work with par</code>, and
borrows should just work as expected. If we are able to bake concurrent and
parallel execution directly into the language, we no longer have to represent
the computation as a type. By making .par</code> a modifier to .await</code>, parallel
futures would not be represented in the type system and we would be able to
solve the scoped parallel async execution problem directly from the language.</strong></p>
Oh and the other nice bonus of this: it would work perfectly with
#[maybe(async)</code>]</a>
function bodies, as we can always fall back from concurrent semantics in async
contexts to sequential semantics in non-async contexts. There is probably also
something interesting to be said about bare par {}</code> blocks and scoped threads,
but that's out of scope (૮꒰ ˶• ༝ •˶꒱ა) for now.</p>
Conclusion</h2>
In this post we've jumped around a fair bit. You would be forgiven for having
some trouble following all threads and ideas. But let me try and summarize the
arguments I've attempted to make:</p>

Tasks are a poor fit for non-parallel concurrent execution.</strong> They come with
additional performance overhead and impose 'static</code> lifetime restrictions,
creating knock-on problems.</li>
Tasks are a poor fit for parallel concurrent execution.</strong> They are presently
designed to function as "async/.await" versions of threads. And as a result
combine both concurrency and parallelism into a single API. Instead we would do
better to provide a ParallelFuture</code> type which provides parallel execution
through composition</em> with concurrency primitives.</li>
The success of the Monoio and Glommio runtimes are putting into question whether
work-stealing executors are the right fit.</strong> They show significant performance
improvements over work-stealing by leveraging a thread-per-core architecture. We
need more data to understand the differences before we can make decisions.</li>
Deciding between thread-per-core and work-stealing executors has
ramifications for what spawn</code> APIs should look like.</strong> Work-stealing executors
can't capture !Send</code> types, while non-work-stealing executors can operate on
!Send</code> types.</li>
There might be benefits to elevating concurrency and parallelism primitives
from the libraries into the language.</strong> Fast, safe, concurrent execution is Rust's
flagship feature, and making that more accessible would likely pay dividends.
This would also provide an alternative way of expressing "scoped tasks".</li>
</ol>
I hope that if you take anything away from this post it's some of the ground
truths of async Rust may be less set in stone than you expected them to be.
There definitely appear to be tradeoffs between the various task scheduling
approaches and designs, and I don't think we should just assume that
work-stealing is the better approach. What we really need is a better
understanding of the tradeoffs involved, and for that we're going to need data.
Deciding without measuring is not going to work.</p>
Oh also: we cannot provide standardized async APIs for parallelism without
first stabilizing async destructors.</strong> I find myself repeating this so often I
feel like a walking meme at this point. Without async destructors so much of
async Rust is broken, and we cannot stabilize interfaces before we know how
async destructors will interact with them. Async parallelism specifically is
also extra-broken</a>.
Async Rust is the flagship feature of priority for Rust as a language, and async
closures + async destructors is where we should be spending our time as they're
fundamental building blocks.</p>
And on a closing note: I just want to put out there that we should dare to
dream beyond the mere ossification of the status quo. Better things are
possible, as long as we take care of each other and are willing to put in the
work. I regularly tell myself this; and now I'm it to you too.</p>

Shipping Jco 1.0, WASI 0.2
Wed, 03 Apr 2024 00:00:00 +0000
On this blog I usually post about technical explorations in programming
languages, API design, asynchronous computing, and so on. More recently I've
re-started my work on WebAssembly, WASI, and their integration with Rust. I
expect that work to eventually become more technical again; but my involvement
there for the past year has mostly been focused on process and people. And I
wanted to take a brief moment to talk about it, because that's the kind of work
that's often easily overlooked. And I'd like to take a little victory lap for a
second (as a treat).</p>
I can proudly say: I helped ship WASI
0.2</a>! Not just a part of it, but
I helped make sure we worked through the last key blocker. And mostly on time
at that 1</a></sup>! And I'm super proud of it! The entire project was an incredible
team effort - it's hard to understate how hard everyone involved has worked on
this. And around five years in the making too. My involvement with the actual
design and implementation was minimal, and I was mostly involved in the second
half of that period. But I can say I've helped in the following ways:</p>
^{1</sup>
When first asked to provide estimates I said something along the lines
of: "I expect we'll be able to ship in late November at the soonest, but most
likely December. December will be tricky because of holidays, but I don't think
we'll slip past January."</em> And Jco ended up being ready in mid-January, with
WASI 0.2 shipping on January 25th. I think that counts as mostly</em> on time,
right?</p>
</div>}
Helped ship Jco 1.0</strong>: Jco is a native JavaScript WebAssembly toolchain and
runtime built for Wasm Components and WASI 0.2. I wrote a blog post about what
it is and how to use it on the Bytecode Alliance
blog</a>. I basically stepped into
the role of PM on that project, ensuring we had clearly defined acceptance
criteria to be able to say: "this thing truthfully implements WASI 0.2". And
then ended up helping make sure we ended up meeting those criteria. This was
my first time donning the PM hat on someone else's project. But that's what
was needed, so I was happy to take that on.</li>
Helped ship WASI 0.2</strong>: WASI 0.2 is a rebase of the WebAssembly System
Interface onto the WebAssembly Components specification. You can read the
announcement post here</a>.
Wasmtime had full support for WASI 0.2, I want to say, around September 2023.
But in order to ratify the actual specification it needed at least two
conforming implementations. My JS days are long behind me; but Jco was the
best candidate to ship a second conforming implementation. So I figured I'd do
what needed to be done, and stepped up to help make sure it shipped. And in
the same meeting we announced Jco had started meeting the conformance
criteria, the WASI subgroup committee voted to ratify the WASI 0.2
specification.</li>
Made the case for a basic async model</strong>: One of the biggest limitations of
WASI 0.1 was that it wasn't really suited to write networked applications with.
I think it was in 2022 that I started making the case that WASI 0.2 probably
should ship with some</em> way to support asynchronous sockets if we wanted
people to actually use it for server programming. And we shouldn't wait for a
future WASI 0.3 to add that potentially years down the line. And that's what we actually
ended up shipping: WASI 0.2 supports async sockets and async HTTP via a
poll-based
interface</a>,
making it possible to write asynchronous networked applications today</em> 2</a></sup>.</li>
Helped ship two new Rust backends</strong>: "new" should be in quotation marks
here - one of the backends is basically a rename. But yeah; I helped ship
support for the renamed wasm32-wasip1</code> target, and the newly introduced
wasm32-wasip2</code> targets. My main contributions here were not so much around
the implementation, but mostly around writing change proposals, talking to
people to help explain the changes and provide assurances, and navigating
various organizational challenges. But it's happened, and bar any unexpected
issues they should be on the stable channels in a few weeks time! Expect a
blog post about this with details on the Rust internals blog soon. (I'm
currently procrastinating finishing that one by writing this instead - sorry
everyone.)</li>
Supported people</strong>: Usually I wouldn't make mention of this as a noteworthy
contribution. But there's talking with people and there's
talking</em> with people. And over the past two years I've spent a non-trivial
amount of time talking with people, listening to what they had to say, and
just trying to be there for them. Wasm and WASI are incredibly ambitious
projects. I'd roughly describe the work as: imagine being tasked
with building several new compilers, defining multiple new operating systems,
creating a mix of novel programming language(s) and a shared ABI. Then take
all of that and make sure it performs at near-native speeds, is able to be
compiled to by virtually every existing programming language, and is usable by non-experts. Then base the
entire thing on a capability-based security model and ensure it is perfectly
sandboxed by default. Finally, all of that technical work gets compacted into
a pressure cooker together with corporate politics, tightening deadlines, and
the pressures of working in open source and open standards. Pandemic.
Layoffs. Burnout. I hope it's not hard to see why that ends up being A Lot.
If there's any takeaway here: sometimes the best open source contribution anyone
can make is just to sit and listen to what people have to say.</li>
</ul>
^2</sup>WASI 0.3's async
model</a> is going to be an enormous
step up from the current status quo though - and timing-wise the work on that
seems to be developing faster than people anticipated. However back in 2022, I
was worried that if WASI 0.2 didn't ship with async support, there would be a
chance adoption would be too slow to make it to WASI 0.3. The state now is that
key industry use cases are unlocked by WASI 0.2, and WASI 0.3 is mainly about
making that work better and faster. And at least to me so far this seems to have
been the right approach.</p>
</div>
And that's it. Unfortunately no cool type theory bits in this post (I like those
a lot; I will</em> talk about type systems for hours if left unchecked) - but I still
wanted to share a little bit about my other recent work. I often catch myself in
the mindset that if I'm not directly coding or designing features, it doesn't
actually count as having worked on something. But that's probably unfair to
myself: I ended up spending several hundred hours doing useful work to help ship
WASI 0.2. And I believe I've actually managed to move the needle on getting WASI
in people's hands sooner. Doing things I wasn't necessarily the most comfortable
with, but it ended up working out! And while I'm not usually one to gloat, I
felt like maybe, perhaps, I do get to gloat a little about this.</p>
I'd also be remiss if I didn't point out that the folks in both the WASI
subgroup committee and in the Bytecode Alliance are such a pleasure to work
with. People regularly talk about how smart they are, and the quality of the
work they produce - but I'd like to instead emphasize just how kind and caring
they are. Talented people, unafraid to dream big, who care about both their
work and each other. WASI 0.2 and Wasm Components are incredibly impressive, and
over the next 6-12 months we're going start seeing the first results of the
impact it will have in the space of applied computing.3</a></sup></p>
^3</sup>To provide an early taste: this
headline</a>
will probably read like marketing hypeware to most people; some company you've
likely never heard of claiming order-of-magnitude improvements in a mature
technology. But the technical work underpinning it is legit. And while I'm not
in a position to vet the exact claims made, they don't at all seem out of line
with what I've seen WASI being able to do so far. I expect similar headlines are
going to keep following now that WASI 0.2 has been ratified and is starting to
be deployed to production. Exciting times!</p>
</div>

Designing an Async Runtime for WASI 0.2
Thu, 29 Feb 2024 00:00:00 +0000
In 2019 Stjepan Glavina and I developed the async-std</code>
runtime</a>. That was an off-shoot from the runtime</code>
project</a>, which itself was an attempt to make it easier
to abstract over different async runtimes. One of the things I'm most proud of
in the work I did on async-std</code> is the core IO abstraction which Stjepan later
factored out into the polling</code></a> and
async-io</code></a> libraries as part of
the smol</code> project</a>, where he took them beyond my initial
prototype code into robust building blocks which can work on their own.</p>
Anyway, that little stroll down memory lane serves a purpose: I just finished
building Yet Another Async
runtime</a>. Not to
tell people to actually go and use this - but intended to be more like a working, minimal, but
also correct implementation of an async runtime for WASI
0.2</a>. The purpose of this post
is to detail how I built it, so
you can build your own (if you want to). I'm one of the first to write this
code, and maybe even the first to write a dedicated runtime, which means that if
Smol</a>,
Monoio</a>,
Glommio</a>, or
Tokio</a> want to add support for WASI 0.2
they're going to have to implement what I already implemented. So I figured I
might save folks some trouble, and document the work I just finished doing.</p>
The WASI 0.2 polling model</h2>
WASI 0.2 is readiness-based rather than completion-based. That means we wait for
the host system to tell us we're ready to take an action, rather than wait for
the host system that an action has successfully completed (completion-based).
WASI 0.3 will likely switch to a completion-based system because Linux
io_uring</code></a> and Windows'
ioringapi</code></a> are
completion-based and perform really well, but we don't have that yet.</p>
The core polling system in WASI consists of two components: the Pollable</code>
type</a> and the
poll</code> function</a>. The
Pollable</code> type represents "interest" in an event. Say we had some kind of
read</code> call; there would be an associated Pollable</code> for that call, which we
could submit to the host to say: "please let me know when there's a reason for
me to call read</code>". This is the "readiness" part of the system - you submit your
interest in an operation to the host system, and then the host system will
periodically yield a list of which operations are good to be called. The way to
schedule that interest is via the poll</code> function.</p>
A key difference between the epoll</code> and WASI 0.2's poll</code> model is that in WASI
we don't schedule interest in resources</em> (e.g. file descriptors), but we
schedule interest in concrete operations. Under epoll</code> we'd tell the kernel
something like: "hey, here's an fd representing some socket - I'm interested
anytime I can read or write new data to it". In WASI we're more precise; we make
a specific method call, which returns a type. That type will have some way to
get the underlying data, but also a method subscribe</code> which returns a pollable.
We're then supposed to wait for the poll</code> call to tell us that the Pollable</code>
is ready - and then we can call the method to get the underlying data without it
returning an error. That's probably a lot of words, so here's a basic example:</p>
use </span>wasi::http::outgoing_handler::{handle, OutgoingRequest};
</span>use </span>wasi::http::types::{Fields, Method, Scheme};
</span>use </span>wasi::io::poll;
</span>
</span>fn </span>main</span>() {
</span>    </span>// Construct an HTTP request
</span>    </span>let</span> fields = Fields::new();
</span>    </span>let</span> req = OutgoingRequest::new(fields);
</span>    req.</span>set_method</span>(&Method::Get).</span>unwrap</span>();
</span>    req.</span>set_scheme</span>(Some(&Scheme::Https)).</span>unwrap</span>();
</span>    req.</span>set_path_with_query</span>(Some("</span>/</span>")).</span>unwrap</span>();
</span>    req.</span>set_authority</span>(Some("</span>example.com</span>")).</span>unwrap</span>();
</span>
</span>    </span>// Send the request and wait for it to complete
</span>    </span>let</span> res = </span>handle</span>(req, None).</span>unwrap</span>();  </span>// 1. We're ready to send the request over the network
</span>    </span>let</span> pollable = res.</span>subscribe</span>();        </span>// 2. We obtain the `Pollable` from the response future
</span>    poll::poll(&[&pollable]);              </span>// 3. Block until we're ready to look at the response
</span>
</span>    </span>// We're now ready to try and access the response headers. If
</span>    </span>// the request was unsuccessful we might still get an error here,
</span>    </span>// but we won't get an error that we tried to read data before the
</span>    </span>// operation was completed.
</span>    </span>let</span> res = res.</span>get</span>().</span>unwrap</span>().</span>unwrap</span>().</span>unwrap</span>();
</span>    </span>for </span>(name, _) in res.</span>headers</span>().</span>entries</span>() {
</span>        println!("</span>header: </span>{name}</span>");
</span>    }
</span>}
</span></code></pre>
When run via
cargo-component</a> (and the
-- -S http</code> flag is passed), this should print the following:</p>
header: age
</span>header: cache-control
</span>header: content-type
</span>header: date
</span>header: etag
</span>header: expires
</span>header: last-modified
</span>header: server
</span>header: vary
</span>header: x-cache
</span>header: content-length
</span></code></pre>
Designing the poller abstraction</h2>
In the HTTP request example we've shown how to make a single HTTP request and
registered that one operation with the poll</code> call. One of the main benefits of
async computing is the ability to perform ad-hoc
concurrency</a>:
we don't just want to be able to wait for the one operation to complete; we want
to be able to wait on any number of operations to complete at the same time</em>.
This is why the poll</code> function takes a list of Pollable</code> types, and returns a
list of which ones are ready to continue doing work.</p>
For that we need to create some type which can hold the pollables for us, be
somewhat efficient about it, and allow us to associate the list of "ready"
events with some notion of identity. This is important because we're building a
runtime for async Rust, which requires that we associate "readiness events" with
"calls to specific
wakers</a>". We can begin by
defining a struct containing a slab</a>. This
is an efficient key-value data structure which on entry gives you a key to
access the value. It is unordered and reuses allocations - which is great
because we'll be continuously registering and deregistering interest in
different Pollable</code>s.</p>
#[</span>derive</span>(Debug)]
</span>pub</span>(</span>crate</span>) </span>struct </span>Poller {
</span>    </span>pub</span>(</span>crate</span>) </span>targets</span>: Slab<Pollable>,
</span>}
</span></code></pre>
Whenever we insert a type into Slab</code> it returns a key of type usize</code>. To make
working with it easier, and make bridging into the WASI world easier, we're
going to define our own EventKey</code> type to wrap the keys for us, like so:</p>
/// A key representing an entry into the poller
</span>#[</span>repr</span>(transparent)]
</span>#[</span>derive</span>(Debug, PartialEq, Eq, PartialOrd, Ord, Hash, Clone, Copy)]
</span>pub</span>(</span>crate</span>) </span>struct </span>EventKey(pub(crate) </span>u32</span>);
</span></code></pre>
Now we're ready to implement the base CRUD methods on the Poller</code>: new</code>,
insert</code>, remove</code>, and get</code>. Aside from the translation to EventKey</code>, there's
nothing interesting going on here.</p>
impl </span>Poller {
</span>    </span>/// Create a new instance of `Poller`
</span>    </span>pub</span>(</span>crate</span>) </span>fn </span>new</span>() -> </span>Self </span>{
</span>        </span>Self </span>{
</span>            targets: Slab::new()
</span>        }
</span>    }
</span>
</span>    </span>/// Insert a new `Pollable` target into `Poller`
</span>    </span>pub</span>(</span>crate</span>) </span>fn </span>insert</span>(&</span>mut </span>self</span>, </span>target</span>: Pollable) -> EventKey {
</span>        </span>let</span> key = </span>self</span>.targets.</span>insert</span>(target);
</span>        EventKey(key as </span>u32</span>)
</span>    }
</span>
</span>    </span>/// Get a `Pollable` if it exists.
</span>    </span>pub</span>(</span>crate</span>) </span>fn </span>get</span>(&</span>self</span>, </span>key</span>: &EventKey) -> Option<&Pollable> {
</span>        </span>self</span>.targets.</span>get</span>(key.</span>0 </span>as </span>usize</span>)
</span>    }
</span>
</span>    </span>/// Remove an instance of `Pollable` from `Poller`.
</span>    </span>///
</span>    </span>/// Returns `None` if no entry was found for `key`.
</span>    </span>pub</span>(</span>crate</span>) </span>fn </span>remove</span>(&</span>mut </span>self</span>, </span>key</span>: EventKey) -> Option<Pollable> {
</span>        </span>self</span>.targets.</span>try_remove</span>(key.</span>0 </span>as </span>usize</span>)
</span>    }
</span>}
</span></code></pre>
And finally we're ready to implement the most interesting part: implementing the
method to wait for events in the Poller</code>, and map them to their respective
EventKey</code>s. When we submit a list of Pollable</code>s to the poll</code> call, we get a
list of indexes</em> back which refer to the indexes of the pollables we submitted.
These are not the same values as the EventKey</code>s we have, so we have to
construct a lookup table to map the indexes of the pollables back to the event
keys.</p>
impl </span>Poller {
</span>    </span>/// Block the current thread until a new event has triggered.
</span>    </span>///
</span>    </span>/// This will clear the value of `ready_list`.
</span>    </span>pub</span>(</span>crate</span>) </span>fn </span>block_until</span>(&</span>mut </span>self</span>) -> Vec<EventKey> {
</span>        </span>// We're about to wait for a number of pollables. When they wake we get
</span>        </span>// the *indexes* back for the pollables whose events were available - so
</span>        </span>// we need to be able to associate the index with the right waker.
</span>
</span>        </span>// We start by iterating over the pollables, and keeping note of which
</span>        </span>// pollable belongs to which waker index
</span>        </span>let mut</span> indexes = Vec::with_capacity(</span>self</span>.targets.</span>len</span>());
</span>        </span>let mut</span> targets = Vec::with_capacity(</span>self</span>.targets.</span>len</span>());
</span>        </span>for </span>(index, target) in </span>self</span>.targets.</span>iter</span>() {
</span>            indexes.</span>push</span>(index);
</span>            targets.</span>push</span>(target);
</span>        }
</span>
</span>        </span>// Now that we have that association, we're ready to poll our targets.
</span>        </span>// This will block until an event has completed.
</span>        </span>let</span> ready_indexes = </span>poll</span>(&targets);
</span>
</span>        </span>// Once we have the indexes for which pollables are available, we need
</span>        </span>// to convert it back to the right keys for the wakers. Earlier we
</span>        </span>// established a positional index -> waker key relationship, so we can
</span>        </span>// go right ahead and perform a lookup there.
</span>        ready_indexes
</span>            .</span>into_iter</span>()
</span>            .</span>map</span>(|</span>index</span>| EventKey(indexes[index as </span>usize</span>] as </span>u32</span>))
</span>            .</span>collect</span>()
</span>    }
</span>}
</span></code></pre>
And with that, our Pollable</code> abstraction is complete!</p>
Designing the reactor abstraction</h2>
Now that we can track lists of pollables and a way to submit them - it's time to
bring wakers into the mix and create a type which can be used by individual
futures to register interest in events. The WASI async runtime is fundamentally
single-threaded, so what we'll want to use here are just the Rc</code> and RefCell</code>
types. We can now construct our data types; again starting with the core structures:</p>
use super</span>::polling::{EventKey, Poller};
</span>
</span>use </span>std::collections::HashMap;
</span>use </span>std::task::Poll;
</span>use </span>std::task::Waker;
</span>use </span>std::{cell::RefCell, rc::Rc};
</span>use </span>wasi::io::poll::Pollable;
</span>
</span>/// Manage async system resources for WASI 0.1
</span>#[</span>derive</span>(Debug, Clone)]
</span>pub struct </span>Reactor {
</span>    </span>inner</span>: Rc<RefCell<InnerReactor>>,
</span>}
</span>
</span>/// The private, internal `Reactor` implementation - factored out so we can take
</span>/// a lock of the whole.
</span>#[</span>derive</span>(Debug)]
</span>struct </span>InnerReactor {
</span>    </span>poller</span>: Poller,
</span>    </span>wakers</span>: HashMap<EventKey, Waker>,
</span>}
</span>
</span>impl </span>Reactor {
</span>    </span>/// Create a new instance of `Reactor`
</span>    </span>pub</span>(</span>crate</span>) </span>fn </span>new</span>() -> </span>Self </span>{
</span>        </span>Self </span>{
</span>            inner: Rc::new(RefCell::new(InnerReactor {
</span>                poller: Poller::new(),
</span>                wakers: HashMap::new(),
</span>            })),
</span>        }
</span>    }
</span>}
</span></code></pre>
The reason we can get away with Rc<RefCell<Inner>></code> is because we control all
the accesses in the methods. We guarantee they are always short-lived and will
never overlap. Which in turn provides us with a convenient way to to be able to
call methods on &self</code> rather than &mut self</code>. Speaking of which, time to
implement the first method - which is a way to register interest in a pollable</code>.</p>
Registering interest in a pollable</h3>
This provides an async method wait_for</code> which allows our reactor to wait
asynchronously until the call related to the pollable</code> is ready to be
performed.</p>
impl </span>Reactor {
</span>    </span>/// Wait for the pollable to resolve.
</span>    </span>pub</span> async </span>fn </span>wait_for</span>(&</span>self</span>, </span>pollable</span>: Pollable) {
</span>        </span>let mut</span> pollable = Some(pollable);
</span>        </span>let mut</span> key = None;
</span>
</span>        </span>// This function is the core loop of our function; it will be called
</span>        </span>// multiple times as the future is resolving.
</span>        future::poll_fn(|</span>cx</span>| {
</span>            </span>// Start by taking a lock on the reactor. This is single-threaded
</span>            </span>// and short-lived, so it will never be contended.
</span>            </span>let mut</span> reactor = </span>self</span>.inner.</span>borrow_mut</span>();
</span>
</span>            </span>// Schedule interest in the `pollable` on the first iteration. On
</span>            </span>// every iteration, register the waker with the reactor.
</span>            </span>let</span> key = key.</span>get_or_insert_with</span>(|| reactor.poller.</span>insert</span>(pollable.</span>take</span>().</span>unwrap</span>()));
</span>            reactor.wakers.</span>insert</span>(*key, cx.</span>waker</span>().</span>clone</span>());
</span>
</span>            </span>// Check whether we're ready or need to keep waiting. If we're
</span>            </span>// ready, we clean up after ourselves.
</span>            </span>if</span> reactor.poller.</span>get</span>(key).</span>unwrap</span>().</span>ready</span>() {
</span>                reactor.poller.</span>remove</span>(*key);
</span>                reactor.wakers.</span>remove</span>(key);
</span>                Poll::Ready(())
</span>            } </span>else </span>{
</span>                Poll::Pending
</span>            }
</span>        })
</span>        .await
</span>    }
</span>}
</span></code></pre>
What's nice about this is that unlike other runtimes, where interest is
registered on the object</em>, here we merely register interest on an operation</em>.
That means that "wait for this operation to be ready" is a matter of calling
reactor.wait_for(pollable).await</code>. That is significantly different than the
juggling of syscalls required in other polling models; and it means we can
encode all of the invariants necessary for the reactor directly as part of the
reactor.</p>
In recent conversations in the Rust WG Async we've been discussing whether it makes sense
to have some kind of reactor hook as part of the Context</code> in future - or
whether perhaps other mechanisms would be better. From this example I believe we
can conclude that while it might be possible</em> to share a reactor through the
future's context - at a high level that will always be mapped the same way to
the underlying function calls. Which at least to me puts into question whether
that is the most natural mapping.</p>
Blocking until events are ready</h3>
With that out of the way, we're ready to implement the wrapper around our
Poller::block_until</code> call. As we implemented, it will wait for all registered
pollers</code>, and give us back a list of EventKey</code>s for the ones which are ready
to be woken. All we have to do is iterate over the list of keys, and call each related waker:</p>
/// Block until new events are ready. Calls the respective wakers once done.
</span>pub</span>(</span>crate</span>) </span>fn </span>block_until</span>(&</span>self</span>) {
</span>    </span>let mut</span> reactor = </span>self</span>.inner.</span>borrow_mut</span>();
</span>    </span>for</span> key in reactor.poller.</span>block_until</span>() {
</span>        </span>match</span> reactor.wakers.</span>get</span>(&key) {
</span>            Some(waker) => waker.</span>wake_by_ref</span>(),
</span>            None => panic!("</span>tried to wake the waker for non-existent `{key:?}`</span>"),
</span>        }
</span>    }
</span>}
</span></code></pre>
At first glance it might seem silly that this goes through the motions
of calling the wakers. WASI 0.2 is currently single-threaded, so by default it
will always just wake the one waker. However, it is common and encouraged to use
wakers to distinguish between events. Concurrency primitives may construct their
own wakers to keep track of identity and wake more precisely. We do not control
the wakers constructed by libraries, and it is for this reason that we have
to call all the wakers - even if by default they will do nothing.</p>
Designing the block_on abstraction</h2>
Now that we have our reactor done, we need a way to drive it. For that we're
going to define a function block_on</code> which takes a closure returning a future
and gives it access to the Reactor</code>. As long as the future returned by the
closure is live, we'll keep making progress on it - waiting on the reactor each
time the future returns Pending</code>.</p>
We'll be driving futures by hand here, so the first step is to define our own
instance of Waker</code>. We're not supporting any form of spawn</code> function, and
instead rely on users to leverage libraries for structured concurrency, so out
of the box we can just make it a noop.</p>
/// Construct a new no-op waker
</span>// NOTE: we can remove this once <https://github.com/rust-lang/rust/issues/98286> lands
</span>fn </span>noop_waker</span>() -> Waker {
</span>    </span>const </span>VTABLE</span>: RawWakerVTable = RawWakerVTable::new(
</span>        </span>// Cloning just returns a new no-op raw waker
</span>        |_| </span>RAW</span>,
</span>        </span>// `wake` does nothing
</span>        |_| {},
</span>        </span>// `wake_by_ref` does nothing
</span>        |_| {},
</span>        </span>// Dropping does nothing as we don't allocate anything
</span>        |_| {},
</span>    );
</span>    </span>const </span>RAW</span>: RawWaker = RawWaker::new(ptr::null(), &</span>VTABLE</span>);
</span>
</span>    </span>// SAFETY: all fields are no-ops, so this is safe
</span>    </span>unsafe </span>{ Waker::from_raw(</span>RAW</span>) }
</span>}
</span></code></pre>
Next up is the actual block_on</code> implementation. The core logic is basically a
loop { match {} }</code> statement which just calls reactor.block_until</code> on each
iteration until the future completes.</p>

</span>/// Start the event loop
</span>pub fn </span>block_on</span><F, Fut>(</span>f</span>: F) -> </span>Fut::</span>Output
</span>where
</span>    F: FnOnce(Reactor) -> Fut,
</span>    Fut: Future,
</span>{
</span>    </span>// Construct the reactor
</span>    </span>let</span> reactor = Reactor::new();
</span>
</span>    </span>// Create the future and pin it so it can be polled
</span>    </span>let</span> fut = (f)(reactor.</span>clone</span>());
</span>    </span>let mut</span> fut = pin!(fut);
</span>
</span>    </span>// Create a new context to be passed to the future.
</span>    </span>let</span> waker = </span>noop_waker</span>();
</span>    </span>let mut</span> cx = Context::from_waker(&waker);
</span>
</span>    </span>// Either the future completes and we return, or some IO is happening
</span>    </span>// and we wait.
</span>    </span>loop </span>{
</span>        </span>match</span> fut.</span>as_mut</span>().</span>poll</span>(&</span>mut</span> cx) {
</span>            Poll::Ready(res) => </span>return</span> res,
</span>            Poll::Pending => reactor.</span>block_until</span>(),
</span>        }
</span>    }
</span>}
</span></code></pre>
And with that, our async runtime is complete!</p>
Using the async runtime</h2>
This section was added post-publication on 2024-03-07</em></p>
Now that we have our runtime, we can use it to wrap the calls from the wasi</code>
library and make them async. For example, say we wanted to have a non-blocking
sleep</code> function which waits for some Duration</code>. We could write that by using
the
subscribe_duration</code></a>
function and connecting it to the reactor. All we need to do is wrap it, call
it, and then wait on the pollable. That should be simple enough to do with what
we've written:</p>
use </span>std::time::Duration;
</span>use </span>wasi::time::monotonic_clock::subscribe_duration;
</span>
</span>pub</span> async </span>fn </span>sleep</span>(</span>duration</span>: Duration, </span>reactor</span>: &Reactor) {
</span>    </span>let</span> duration = duration.</span>as_nanos</span>() as </span>u64</span>;     </span>// 1. Convert the duration to nanosecond resolution
</span>    </span>let</span> pollable = </span>subscribe_duration</span>(duration);   </span>// 2. Obtain the pollable
</span>    reactor.</span>wait_for</span>(pollable).await;              </span>// 3. Wait for the pollable to be ready
</span>}
</span></code></pre>
It should be simple enough to take APIs from the wasi</code> crate, and pair them
with a call to reactor.wait_for(..).await</code>. That's the base pattern we's use to
asyncify basically the entire wasi</code> API surface.</p>
On the absence of a local executor</h2>
WASI 0.2 is single-threaded, and fundamentally doesn't need access to a
task.spawn</code> abstraction. In sync Rust we use threads to combine parallelism +
concurrency; but in async Rust we can separate the two. The
futures-concurrency</a> library provides access to any mode of concurrency you
might want; meaning that in the absence of parallelism there is no reason for an
executor to exist.</p>
Instead APIs such as
Vec::join</code></a>
and
StreamGroup</code></a>
even provide access to the unbounded concurrency primitives people typically use
"local executors" for - but without any of the scoped lifetime issues plaguing
those abstractions. Here is an example of how to concurrently make two separate
HTTP requests without having to rely on "local tasks":</p>
fn </span>main</span>() {
</span>    </span>block_on</span>(|</span>reactor</span>| async {
</span>        </span>let</span> client = Client::new(reactor); </span>// <- using an unpublished wrapper around `wasi::http`
</span>
</span>        </span>let</span> a = async {
</span>            </span>let</span> url = "</span>https://example.com</span>".</span>parse</span>().</span>unwrap</span>();
</span>            </span>let</span> req = Request::new(Method::Get, url);
</span>            </span>let</span> res = client.</span>send</span>(req).await;
</span>
</span>            </span>let</span> body = </span>read_to_end</span>(res).await;
</span>            </span>let</span> body = String::from_utf8(body).</span>unwrap</span>();
</span>            println!("</span>{body}</span>");
</span>        };
</span>
</span>        </span>let</span> b = async {
</span>            </span>let</span> url = "</span>https://example.com</span>".</span>parse</span>().</span>unwrap</span>();
</span>            </span>let</span> req = Request::new(Method::Get, url);
</span>            </span>let</span> res = client.</span>send</span>(req).await;
</span>
</span>            </span>let</span> body = </span>read_to_end</span>(res).await;
</span>            </span>let</span> body = String::from_utf8(body).</span>unwrap</span>();
</span>            println!("</span>{body}</span>");
</span>        };
</span>
</span>        (a, b).</span>join</span>().await; </span>// concurrently await both `a` and `b`.
</span>    })
</span>}
</span></code></pre>
Conclusion</h2>
In this post I've explained WASI's polling model and showed step-by-step how to
use it to build your own async runtime. I hope this will be useful for
maintainers of async Rust runtimes, as well as hobbyists wanting to go through
the motions of writing their own. If you prefer to just use the code I've shared
here today, you can do so by installing the wasi-async-runtime</code>
crate</a>.</p>

Extending Rust's Effect System
Fri, 09 Feb 2024 00:00:00 +0000
</iframe>
This is the transcript of my RustConf 2023 talk: "Extending Rust's Effect
System", presented on September 13th 2023 in Albuquerque, New Mexico and
streamed online.</em></p>
Introduction</h2>
Rust has continuously evolved since version 1.0 was released in 2015. We've
added major features such as the try operator (?</code>), const generics, generic
associated types (GATs), and of course: async/.await</code>. Out of those four
features, three are what can be considered to be "effects". And though we've
been working on them for a long time, they are all still very much in-progress.</p>
Hello, my name is Yosh and I work as a Developer Advocate for Rust at Microsoft.
I've been working on Rust itself for the past five years, and I'm among other
things a member of the Rust Async WG, and the co-lead of the Rust Effects
Initiative.</p>
The thesis of this talk is that we've unknowingly shipped an effect system as
part of the language in since Rust 1.0. We've since begun adding a number of new
effects, and in order to finish integrating them into the language we need
support for effect generics</em>.</p>
In this talk I'll explain what effects are, what makes them challenging to
integrate into the language, and how we can overcome those challenges.</p>
Rust Without Generics</h2>
When I was new to Rust it took me a minute to figure out how to use generics. I
was used to writing JavaScript, and we don’t have generics there. So I found
myself mostly writing functions which operated on concrete types</em>. I remember
my code felt pretty clumsy, and it wasn't a great experience. Not compared to,
say, the code the stdlib provides.</p>
An example of a generic stdlib function is the io::copy</code> function. It reads
bytes from a reader, and copies them into a writer. We can give it a file, a
socket, or any combination of the two, and it will happily copy bytes from one
into the other.  This all works as long as we give it the right types.</p>
But what if Rust actually didn't have generics? What if the Rust I used to write
at the beginning was actually all we had? How would we write this io::copy</code>
function? Well, given we're trying to copy bytes between sockets and file types,
we could probably hand-code individual functions for these. For our two types
here we could write four unique functions.</p>
But unfortunately for us that would only solve the problem right in front of us.
But the stdlib doesn’t just have two types which implement read and write. It
has 18 types which implement read, and 27 types which implement write. So if we
wanted to cover the entire API space of the stdlib, we’d need 486 functions in
total. And if that was the only way we could implement io::copy</code>, that would
make for a pretty bad language.</p>
Now luckily Rust does have generics, and all we ever need is the one copy</code>
function. This means we're free to keep adding more types into the stdlib
without having to worry about implementing more functions. We just have the one
copy</code> function, and the compiler will take care of generating the right code
for any types we give it.</p>
Why effect generics?</h2>
Types are not the only things in Rust we want to be generic over. We also have
"const generics" which allow functions to be generic over constant values. As
well as "value generics" which allow functions to be generic over different
values. This is how we can write functions which can take different values -
which is a feature present in most programming languages.</p>
fn </span>by_value</span>(</span>cat</span>: Cat) { .. }
</span>fn </span>by_reference</span>(</span>cat</span>: &Cat) { .. }
</span>fn </span>by_mutable_reference</span>(</span>cat</span>: &</span>mut</span> Cat) { .. }
</span></code></pre>
But not everything that can lead to API duplication are things we can be generic
over. For example, it's pretty common to create different methods or types
depending on whether we take a value as owned, as a reference, or as a mutable
reference. We also often create duplicate APIs for constant values and for
runtime values. As well as create duplicate structures depending on whether the
API needs to be thread-safe or not.</p>
But out of everything which can lead to API duplication, effects are probably
one of the biggest ones. When I talk about effects in Rust, what I mean is
certain keywords such as async/.await</code> and const</code>; but also ?</code>, and types
such as Result</code>, and Option</code>. All of these have a deep, semantic connection to
the language, and changes the meaning of our code in ways that other keywords
and types don't.</p>
Sometimes we'll write code which doesn't have the right effects, leading to
effect mismatches</em>. This is also known as the function coloring problem</em>, as
described by Robert Nystrom. Once you become aware of effect mismatches</em> you
start seeing them all over the place, not just in Rust either.</p>
The result of these effect mismatches is that using effects in Rust essentially
drops you into a second-rate experience. Whether you're using const, async,
Result, or Error - almost certainly somewhere along the line you'll run into a
compatibility issue.</p>
let</span> db: Option<Database> = ..;
</span>let</span> db = db.</span>filter</span>(|</span>db</span>| db.name? == "</span>chashu</span>");
</span></code></pre>
Take for example the Option::filter</code> API. It takes a type by reference and
returns a bool. If we try and use the ?</code> operator inside of it we get an error,
because the function doesn't return Result</code> or Option</code>. Not being able to use
?</code> inside of arbitrary closures is an example of an effect mismatch.</p>
But simple functions like that only scratch the surface. Effect mismatches are
present in almost every single trait in the stdlib too. Take for example
something common like the Debug</code> trait which is implemented on almost every
type in Rust.</p>
We could implement the Debug</code> trait for our made-up type Cat</code>. The parameter
f</code> here implements io::Write</code> and represents a stream of bytes. And using the
write!</code> macro we can write bytes into that stream. But if for some reason we
wanted to write bytes asynchronously into, say, an async socket. Well, we can't
do that. fn fmt</code> is not an async function, which means we can't await inside of
it.</p>
One way out of this could be to create some kind of intermediate buffer, and
synchronously write data into it. We could then write data out of that buffer
asynchronously. But that would involve extra copies we didn't have before.</p>
If we wanted to make it identical to what we did before, the solution would be
to create a new</em> AsyncDebug</code> trait which can</em> write data asynchronously into
the stream. But we now have duplicate traits, and that's exactly the problem
we're trying to prevent.</p>
It's tempting to say that maybe we should just add the AsyncDebug</code> trait and
call it a day. We can then also add async versions of Read</code>, Write</code>, and
Iterator</code> too. And perhaps Hash</code> as well, since it too writes to an output
stream. And what about From</code> and Into</code>? Perhaps Fn</code>, FnOnce</code>, FnMut</code>, and
Drop</code> too since they're built-ins? And so on. The reality is that effect
mismatches are structural, and duplicating the API surface for every effect
mismatch leads to an exponential explosion of APIs. Which is similar to what
we've seen with data type generics earlier on.</p>
Let me try and illustrate this for a second. Say we took the existing family of
Fn</code> traits and introduced effectful versions of them. That is: versions which
work with unsafe</code> 1</a></sup>, async</code>, try</code>, const</code>, and generators. With one effect
we're up to six unique traits. With two effects we're up to twelve. With all
five we're suddenly looking at 96 different traits.</p>
^{1</sup>
Correction from 2024: after having discussed this with Ralf Jung
we've concluded that semantically unsafe</code> in Rust is not an effect. But
syntactically it would be fair to say that unsafe</code> is "effect-like". As such
any notion of "maybe-unsafe" would be nonsensical. We don't discuss such a
feature in this talk, but it is worth clearing up ahead of time in case this
leaves people wondering.</p>
</div>
The problem space in the stdlib is really broad. From analyzing the Rust 1.70
stdlib, by my estimate about 75% of the stdlib would interact with the const
effect. Around 65% would interact with the async effect. And around 30% would
interact with the try effect. The exact numbers are imprecise because parts of
the various effects are still in-progress. How much this will result in
practice, very much will depend on how we end up designing the language.</p>
If you compare the numbers then it appears that close to 100% of the stdlib
would interact with one or more effect. And about 50% would interact with two or
more effects. If we consider that whenever effects interact with each other they
can lead to exponential blowup, this should warn us that clever one-off
solutions won't cut it. I believe that the best way to deal with this is to
instead allow Rust to enable items to be generic over effects.</p>
Stage I: Effect-Generic Trait Definitions</h2>
Now that we've taken a good look at what happens when we can't be generic over
effects, it's time we start talking about what we can do about it. The answer,
unsurprisingly, is to introduce effect generics into the language. To cover all
uses will take a few steps, so let's start with the first, and arguably most
important one: effect-generic trait definitions.</p>
This is important because it would allow us to introduce effectful traits as
part of the stdlib. Which would among other things would help standardize the
various async ecosystems around the stdlib.</p>
pub trait </span>Into<T>: Sized {     
</span>    </span>fn </span>into</span>(</span>self</span>) -> T;
</span>}
</span></code></pre>
impl </span>Into<Loaf> </span>for </span>Cat {     
</span>    </span>fn </span>into</span>(</span>self</span>) -> Loaf {
</span>        </span>self</span>.</span>nap</span>()
</span>    }
</span>}
</span></code></pre>
Let's use a simple example here: the Into</code> trait. The Into</code> trait is used to
convert from one type into another. It is generic over a type T</code>, and has one
function "into" which consumes Self</code> and returns the type T</code>. Say we have a
type cat which when it takes a nap turns into a cute little loaf. We can
implement Into<Loaf></code> for Cat</code> by calling self.nap</code> in the function body.</p>
pub trait </span>AsyncInto<T>: Sized {     
</span>    async </span>fn </span>into</span>(</span>self</span>) -> T;
</span>}
</span></code></pre>
impl </span>AsyncInto<Loaf> </span>for </span>Cat {     
</span>    async </span>fn </span>into</span>(</span>self</span>) -> Loaf {
</span>        </span>self</span>.</span>nap</span>().await
</span>    }
</span>}
</span></code></pre>
But what if the cat doesn't take a nap straight away? Maybe nap</code> should
actually be an async function. In order to await</code> nap inside the trait impl,
the into</code> method would need to be async. If we were writing an async trait from
scratch, we could do this by exposing a new AsyncInto</code> trait with an async
into</code> method.</p>
But we don't just want to add a new trait to the stdlib, instead we want to
extend</em> the existing Into</code> trait to work with the async effect. The way we
could extend the Into</code> trait with the async</code> effect is by making the async
effect optional</em>. Rather than requiring that the trait is always sync or async,
implementors should be able to choose which version of the trait they want to
implement.</p>
#[</span>maybe</span>(async)]
</span>impl </span>Into<Loaf> </span>for </span>Cat {     
</span>    #[</span>maybe</span>(async)]
</span>    </span>fn </span>into</span>(</span>self</span>) -> Loaf {
</span>        </span>self</span>.</span>nap</span>()
</span>    }
</span>}
</span></code></pre>
The way this would work is by adding a new notation on the trait: "maybe async".
We don't yet know what syntax we want to use for "maybe async", so in this talk
we'll be using attributes. The way the "maybe async" notation works is that we
mark all methods which we want to be "maybe async" as such. And then mark our
trait itself as "maybe async" too.</p>
impl </span>Into<Loaf> </span>for </span>Cat {     
</span>    </span>fn </span>into</span>(</span>self</span>) -> Loaf {
</span>        </span>self</span>.</span>nap</span>()
</span>    }
</span>}
</span></code></pre>
impl </span>async Into<Loaf> for Cat {     
</span>    async </span>fn </span>into</span>(</span>self</span>) -> Loaf {
</span>        </span>self</span>.</span>nap</span>().await
</span>    }
</span>}
</span></code></pre>
Implementors then get to choose whether they want to implement the sync or async
versions of the trait. And depending on which version they choose, the methods
then ends up being either sync or async. This system would be entirely
backwards-compatible, because implementing the sync version of Into</code> would
remain the same as it is today. But people who want to implement the async
version would be able to, just by adding a few extra async</code> keywords to the impl.</p>
impl </span>async Into<Loaf> for Cat {
</span>    async </span>fn </span>into</span>(</span>self</span>) -> Loaf {
</span>        </span>self</span>.</span>nap</span>().await
</span>    }
</span>}
</span></code></pre>
impl </span>Into<Loaf, true> </span>for </span>Cat {
</span>    </span>type </span>ReturnTy = </span>impl </span>Future<Output = Loaf>;
</span>    fn into(self) -> Self::ReturnTy {
</span>        async </span>move </span>{
</span>            </span>self</span>.</span>nap</span>().await
</span>        }
</span>    }
</span>}
</span></code></pre>
Under the hood the implementations desugars to regular Rust code we can already
write today. The sync implementation of the type returns a type T</code>. But the
async impl returns an impl Future</code> of T</code>. Under the hood it is just a single
const bool and some associated types.</p>

good diagnostics</li>
gradual stabilization,</li>
backwards-compatibility</li>
clear inference rules</li>
</ul>
It would be reasonable to ask why we're bothering with a language feature, if
the desugaring ends up being so simple. And the reason is: effects are
everywhere, and we want to make sure effect generics feel like part of the
language. That not only means that we want to tightly control the diagnostics.
We also want to enable them to be gradually introduced, have clear language
rules, and be backwards-compatible.</p>
But if you keep all that in mind, it's probably okay to think of effect generics
as mostly syntactic sugar for const bools + associated types.</p>
Stage II: Effect-Generic Bounds, Impls, and Types</h2>
Being able to declare effect-generic traits is only the beginning. The stdlib
not only exposes traits, it also exposes various types and functions. And
effect-generic traits don't directly help with that.</p>
pub fn </span>copy</span><R, W>(
</span>    </span>reader</span>: &</span>mut</span> R,
</span>    </span>writer</span>: &</span>mut</span> W
</span>) -> io::Result<()>
</span>where
</span>    R: Read,
</span>    W: Write;
</span></code></pre>
Let's take our earlier io::copy</code> example again. As we've said copy</code> takes a
reader and writer, and then copies bytes from the reader to the writer. We've
seen this.</p>
pub fn </span>async_copy</span><R, W>(
</span>    </span>reader</span>: &</span>mut</span> R,
</span>    </span>writer</span>: &</span>mut</span> W
</span>) -> io::Result<()>
</span>where
</span>    R: AsyncRead,
</span>    W: AsyncWrite;
</span></code></pre>
Now what would it look like if we tried adding an async version of this to the
stdlib today. Well, we'd need to start by giving it a different name so it
doesn't conflict with the existing copy</code> function. The same goes for the trait
bounds as well, so instead of taking Read</code> and Write</code>, this function would
take AsyncRead</code> and `AsyncWrite.</p>
pub fn </span>async_copy</span><R, W>(
</span>    </span>reader</span>: &</span>mut</span> R,
</span>    </span>writer</span>: &</span>mut</span> W
</span>) -> io::Result<()>
</span>where
</span>    R: async Read,
</span>    W: async Write;
</span></code></pre>
Now things get a little better once we have effect-generic trait definitions.
Rather than needing to take async duplicates of the Read</code> and Write</code> traits,
the function can instead choose the async versions of the existing Read</code> and
Write</code> traits. That's already better, but it still means we have two versions
of the copy</code> function.</p>
#[</span>maybe</span>(async)]
</span>pub fn </span>copy</span><R, W>(
</span>    </span>reader</span>: &</span>mut</span> R,
</span>    </span>writer</span>: &</span>mut</span> W
</span>) -> io::Result<()>
</span>where
</span>    R: #[maybe(async)] Read,
</span>    W: #[maybe(async)] Write;
</span></code></pre>
Instead the ideal solution would be to allow copy</code> itself to be generic over
the async effect, and make that determine which versions of Read</code> and Write</code>
we want. These are what we call "effect-generic bounds". The effect of the
function and the effect of the bounds it takes all become the same. In
literature this is also known as "row-polymorphism".</p>
copy</span>(reader, writer)?;                </span>// infer sync
</span>copy</span>(reader, writer).await?;          </span>// infer async
</span>copy::<async>(reader, writer).await?; </span>// force async
</span></code></pre>
Because the function itself is now generic over the async effect, we need to
figure out at the call-site which variant we intended to use. This system will
make use of inference</em> to figure it out. That's a fancy way of saying that
the compiler is going to make an educated guess about which effects the
programmer intended to use. If they used .await</code> they probably wanted the async
version. Otherwise they probably wanted the sync version. But as with any guess:
sometimes we guess wrong, so for that reason we want to provide an escape hatch
by enabling program authors to force the variant. We don't know the exact syntax
for this yet, but we assume this would likely be using the turbofish notation.</p>
struct </span>File { .. }
</span>impl </span>File {
</span>    </span>fn </span>open</span><P>(</span>p</span>: P) -> Result<</span>Self</span>>
</span>    </span>where
</span>        P: AsRef<Path>;
</span>}
</span></code></pre>
But effect-generics aren't just needed for functions. If we want to make the
stdlib work well with effects, then types will need effect-generics too. This
might seem strange at first, since an "async type" might not be very intuitive.
But for example files on Windows need to be initialized as either sync or async.
Which means that whether they're async or not isn't just a property of the
functions, it's a property of the type.</p>
Let's use the stdlib's File</code> type as our example here. For simplicity let's
assume it has a single method: open</code> which returns either an error or a file.</p>
struct </span>AsyncFile { .. }
</span>impl </span>AsyncFile {
</span>    async </span>fn </span>open</span><P>(</span>p</span>: P) -> Result<</span>Self</span>>
</span>    </span>where
</span>        P: AsRef<AsyncPath>;
</span>}
</span></code></pre>
If we wanted to provide an async version of File</code>, we again would need to
duplicate our interfaces. That means a new type AsyncFile</code>, which has a new
async method open</code>, which takes an async version of Path</code> as an argument. And
Path</code> needs to be async because it itself has async filesystem methods on it.
As I've said before: once you start looking you notice effects popping up
everywhere.</p>
#[</span>maybe</span>(async)]
</span>struct </span>File { .. }
</span>
</span>#[</span>maybe</span>(async)] 
</span>impl </span>File {
</span>    #[</span>maybe</span>(async)]
</span>    </span>fn </span>open</span><P>(</span>p</span>: P) -> Result<</span>Self</span>>
</span>    </span>where
</span>        P: AsRef<</span>#</span>[maybe(async)] Path>;
</span>}
</span></code></pre>
Instead of creating a second AsyncFile</code> type, with effect generics on types
we'd be able to open File</code> as async instead. Allowing us to keep just the one
File</code> definition for both sync and async variants.</p>
#[</span>maybe</span>(async)]
</span>fn </span>copy</span><R, W>(</span>reader</span>: R, </span>writer</span>: W) -> io::Result<()> {
</span>    </span>let mut</span> buf = vec![</span>4028</span>];
</span>    </span>loop </span>{
</span>        </span>match</span> reader.</span>read</span>(&</span>mut</span> buf).await? {
</span>            </span>0 </span>=> </span>return </span>Ok(()),
</span>            n => writer.</span>write_all</span>(&buf[</span>0</span>..n]).await?,
</span>        }
</span>    }
</span>}
</span></code></pre>
Now I've sort of hand-waved away the internal implementations of both the copy</code>
function and the File</code> type. The way they work is a little different for the
two. In the case of the copy</code> function, the implementation between the async
and non-async variants would be identical. If the function is compiled as async,
everything works as written. But if the function compiles as sync, then we just
remove the .await</code>s and the function should compile as expected.</p>
As a result of this "maybe-async" functions can only call sync or other
"maybe-async" functions. But that should be fine for most cases.</p>
impl </span>File {
</span>    #[</span>maybe</span>(async)]
</span>    </span>fn </span>open</span><P>(</span>p</span>: P) -> Result<</span>Self</span>> {
</span>        </span>if </span>IS_ASYNC </span>{ .. } </span>else </span>{ .. }
</span>    }
</span>}
</span></code></pre>
Concrete types like File</code> are a little trickier. They often want to run
different code depending on which effects it has. Luckily types like File</code>
already conditionally compile different code depending on the platform, so
introducing new types conditions shouldn't be too big of a jump. The key thing
we need is a way to detect in the function body whether code is being compiled
as async or not - basically a fancy bool.</p>
We can already do this for the const effect using the const_eval_select</code>
intrinsic. It's currently unstable and a little verbose, but it works reliably.
We should be able to easily adapt it to something similar for async and the rest
of the effects too.</p>
What are effects?</h2>
Systems research on effects has been a topic in computer science for nearly 40
years. That's about as old as C++. It's become a bit of a hot topic recently in
PL spheres with research languages such as Koka, Eff, and Frank showing how
effects can be useful. And languages such as Scala, and to a lesser extent
Swift, adopting effect features.</p>
When people talk about effects they will broadly refer to one of two things:</p>

Algebraic Effect Types:</strong> which are semantic notations on functions and contexts
that grant a permission to do</em> something.</li>
Algebraic Effect Handlers:</strong> which are a kind of typed control-flow
primitive which allows people to define their own versions of async/.await</code>,
try..catch</code>, and yield</code>.</li>
</ul>
A lot of languages which have effects provide both effect types and effect
handlers. These can be used together, but they are in fact distinct features. In
this talk we'll only be discussing effect types.</p>
pub</span> async </span>fn </span>meow</span>(</span>self</span>) {}
</span>pub const unsafe fn </span>meow</span>() {}
</span></code></pre>
What we've been calling "effects" in this talk so far have in fact been effect
types</em>. Rust hasn't historically called them this, and I believe that's probably
why effect generics weren't on our radar until recently. But it turns out that
reinterpreting some of our keywords as effect types actually makes perfect
sense, and provides us with a strong theoretical framework for how to reason
about them.</p>
We also have unsafe</code> which allows you to call unsafe</code> functions. The unstable
try-block feature which doesn't require you to Ok</code>-wrap return types. The
unstable generator closure syntax which gives you access to the yield</code> keyword.
And of course the const</code> keyword which allows you evaluate code at
compile-time.</p>
async { </span>async_fn</span>().await }; </span>// async effect
</span>unsafe </span>{ </span>unsafe_fn</span>() };     </span>// unsafe effect
</span>const </span>{ </span>const_fn</span>() };       </span>// const effect
</span>try { </span>try_fn</span>()? };          </span>// try effect (unstable)
</span>|| { </span>yield</span> my_type };       </span>// generator effect (unstable)
</span></code></pre>
In Rust we currently have five different effects: async</code>, unsafe</code>, const</code>,
try</code>, and generators. All six of these are in various stages of completion. For
example: async Rust has functions and blocks, but no iterators or drop. Const
doesn't have access to traits yet. Unsafe functions can't be lowered to Fn</code>
traits. Try does have the ?</code> operator, but try blocks are unstable. And
generators are entirely unstable; we only have the Iterator</code> trait.</p>
Some of these effects are what folks on the lang team have started calling
"carried". Those are effects which will desugar to an actual type in the type
system. For example when you write async fn</code>, the return type will desugar to
an impl Future</code>.</p>
Some other effects are what we're calling: "uncarried"</em>. These effects don't desugar
to any types in the type system, but serve only as a way to communicate
information back to the compiler. This is for example const</code> or unsafe</code>. While
we do check that the effects are used correctly, they don't end up being lowered
to actual types.</p>
let</span> x = try async { .. };
</span></code></pre>
1. -> impl Future<Output = Result<T, E>>
</span>2. -> Result<impl Future<Output = T>, E>
</span></code></pre>
When we talk about carried effects, effect composition becomes important. Take
for example "async" and "try" together. If we have a function which has both?
What should the resulting type be? A future of Result? Or a Result containing a
Future?</p>
Effects on functions are order-independent sets</em>. While Rust currently does
require you declare effects in a specific order, carried effects themselves can
only be composed in one way. When we stabilized async/.await</code>, we decided that
if an async function returned a Result, that should always return an impl Future</code> of Result</code>.  And because effects are sets</em> and not dependent on
ordering, we can define the way carried effects should compose as part of the
language.</p>
People can still opt-out from the built-in composition rules by manually writing
function signatures. But this is rare, and for the overwhelming majority of uses
the built-in composition rules will be the right choice.</p>
const fn </span>meow</span>() {}  </span>// maybe-const
</span>const </span>{}            </span>// always-const
</span></code></pre>
The const</code> effect is a bit different from the other effects. const</code> blocks are
always</em> evaluated during compilation. While const</code> functions merely can</em> be
evaluated during during compilation. It's perfectly fine to call them at runtime
too. This means that when we write const fn</code>, we're already writing
effect-generics. This mechanism is the reason why we've gradually been able to
introduce const into the stdlib in a backwards-compatible way.</p>
Const is also a bit strange in that among other things it disallows access to
the host runtime, it can't allocate, and it can't access globals. This feels
different from effects like say, async</code>, which only allow you to do more
things.</p>
effect set</th> can access</th> cannot access</th></tr></thead>

std rust</td> non-termination, unwinding, non-determinism, statics, runtime heap, host APIs</td> N/A</td></tr>
alloc</td> non-termination, unwinding, non-determinism, globals, runtime heap</td> host APIs</td></tr>
core</td> non-termination, unwinding, non-determinism, globals</td> runtime heap, host APIs</td></tr>
const</td> non-termination, unwinding</td> non-determinism, globals, runtime heap, host APIs</td></tr>
</tbody></table>
What's missing from this picture is that all functions in Rust carry an implicit
set of effects. Including some effects we can't directly name yet. When we write
const</code> functions, our functions have a different set of effects, than if we
write no_std</code> functions, which again are different from regular "std" rust
functions.</p>
The right way of thinking about const, std, etc. is as adding a different
effects to the empty set of effects. If we start from zero, then all effects are
merely additive. They just add up to different numbers.</p>
Unfortunately in Rust we can't yet name the empty set of effects. In effect
theory this is called the "total effect". And some languages such as Koka do
support the "total" effect. In fact, Koka's lead developer has estimated that
around 70% of a typical Koka program can be total. Which begs the question: if
we could express the total effect in Rust, could we see similar numbers?</p>
Stage III: More Effects</h2>
So far we've only talked about how we could finish the work on existing effects
such as const</code> and async</code>. But one nice thing of effect generics is that they
would not only allow us to finish our ongoing effects work. It would also lower
the cost of introducing new</em> effects to the language.</p>
Which opens up the question: if we could add more effects, which effects might
make sense to add? The obvious ones would be to actually finish adding try</code> and
generator functions. But beyond that, there are some interesting effects we
could explore. For brevity I'll only discuss what these features are, and not
show code examples.</p>

no-divergence</strong>: guarantees that a function cannot loop indefinitely,
opening up the ability to perform static runtime-cost analysis.</li>
no-panic</strong>: guarantees a function will never produce a panic, causing
the function to unwind.</li>
parametricity</strong>: guarantees that a function only operates on its arguments. That
means no implicit access to statics, no global filesystem, no thread-locals.</li>
capability-safety</strong>: guarantees that a function is not only parametric, but can't
downcast abstract types either. Say if you get an impl Read</code>, you can't reverse
it to obtain a File</code>.</li>
destructor linearity</strong>: guarantees that Drop</code> will always</em> be called,
making it a safety guarantee.</li>
pattern types</strong>: enables functions to operate directly on variants of enums
and numbers</li>
must-not-move types</strong>: would be a generalization of pinning and the
pin-project system, making it a first-class language feature</li>
</ul>
Though there's nothing inherently stopping us from adding any of these features
into Rust today, in order to integrate them into the stdlib without breaking
backwards-compatibility we need effect generics first.</p>
effect </span>const  </span>= diverge + panic;
</span>effect core   = </span>const </span>+ statics + non_determinism;
</span>effect alloc  = core + heap;
</span>effect std    = alloc + host_apis;
</span></code></pre>
This brings us to the final part of the design space: effect aliases. If we keep
adding effects it's very easy to eventually reach into a situation where we have
our own version of "public static void main".</p>
In order to mitigate that it would instead be great if we could name specific
sets of effects. In a way we've already done that, where const</code> represents "may
loop forever" and "may panic". If we actually had "may loop forever" and "may
panic" as built-in effects, then we could redefine const</code> as an alias to those.</p>
Fundamentally this doesn't change anything we've talked about so far. It's just
that this would syntactically be a lot more pleasant to work with. So if we ever
reach a state where we have effect generics and we want notice we maybe have one
too many notation in front of our functions, it may be time for us to start
looking into this more seriously.</p>
Outro</h2>
Rust already includes effect types such as async, const, try, and unsafe.
Because we can't be generic over effect types yet, we usually have to choose
between either duplicating code, or just not addressing the use case. And this
makes for a language which feels incredibly rough once you start using effects.
Effect generics provide us with a way to be generic over effects, and we've
shown they can be implemented today as mostly as syntax sugar over
const-generics.</p>
We're currently in the process of formalizing the effect generic work via the
A-Mir-Formality. MIR Formality is an in-progress formal model of Rust's type
system. Because effect generics are relatively straight forward but have
far-reaching consequences for the type system, it is an ideal candidate to test
as part of the formal model.</p>
In parallel the const WG has also begun refactoring the way const functions are
checked in the compiler. In the past const-checking happened right before borrow
checking at the MIR level. In the new system const-checking will happen much
sooner, at the HIR level. This will not only make the code more maintainable, it
will also be generalizable to more effects if needed.</p>
Once both the formal modeling and compiler refactorings conclude, we'll begin
drafting an RFC for effect-generic trait definitions. We expect this to happen
sometime in 2024.</p>
And that's the end of this talk. Thank you so much for being with me all the way
to the end. None of the work in this talk would have been possible without the
following people:</p>

Oliver Scherer (AWS)</li>
Eric Holk (Microsoft)</li>
Niko Matsakis (AWS)</li>
Daan Leijen (Microsoft)</li>
</ul>
Thank you!</p>

Will it block?
Wed, 07 Feb 2024 00:00:00 +0000
Will this function block?</p>
fn </span>work</span>(</span>x</span>: </span>u64</span>, </span>y</span>: </span>u64</span>) -> </span>u64 </span>{
</span>    x + y
</span>}
</span></code></pre>
How about this function?</p>
fn </span>work</span>(</span>x</span>: </span>u32</span>) -> </span>u32 </span>{
</span>    </span>let mut</span> prev = </span>0</span>;
</span>    </span>let mut</span> curr = </span>1</span>;
</span>    </span>for </span>_ in </span>0</span>..x {
</span>        </span>let</span> next = prev + curr;
</span>        prev = curr;
</span>        curr = next;
</span>    }
</span>    curr
</span>}
</span></code></pre>
Or this one?</p>
use </span>sha256::digest;
</span>
</span>fn </span>work</span>(</span>input</span>: &</span>str</span>) -> String {
</span>    </span>digest</span>(input)
</span>}
</span></code></pre>
Will this function block?</p>
use </span>std::{thread, time::Duration};
</span>fn </span>work</span>() {
</span>    thread::sleep(Duration::from_nanos(</span>10</span>));
</span>}
</span></code></pre>
Perhaps this one will?</p>
use </span>std::{thread, time::Duration};
</span>fn </span>work</span>() {
</span>    thread::sleep(Duration::from_millis(</span>10</span>));
</span>}
</span></code></pre>
Or how about this one, does this block?</p>
use </span>std::{thread, time::Duration};
</span>fn </span>work</span>() {
</span>    thread::sleep(Duration::from_secs(</span>10</span>));
</span>}
</span></code></pre>
Let's try another; does this block?</p>
use </span>std::{thread, time::Duration};
</span>fn </span>work</span>() {
</span>    thread::spawn(|| {}).</span>join</span>();
</span>}
</span></code></pre>
Up next: does this block?</p>
use </span>std::{thread, time::Duration};
</span>fn </span>work</span>() {
</span>    println!("</span>meow</span>");
</span>}
</span></code></pre>
Last one; does this block?</p>
use </span>std::fs::File;
</span>use </span>std::io::prelude::*;
</span>use </span>std::os::unix::io::FromRawFd;
</span>
</span>fn </span>work</span>() {
</span>    </span>let mut</span> f = </span>unsafe </span>{ File::from_raw_fd(</span>1</span>) };
</span>    write!(&mut f, "</span>meow</span>").</span>unwrap</span>();
</span>}
</span></code></pre>
And so on.</p>
Determining whether something "is blocking" is a messy endeavor because
computers are messy. Whether something blocks is not something we can
objectively determine, but depends on our own definitions and uses. A system
call which resolves quickly might not be considered "blocking" at all
^{1</a></sup>, while a computation which takes a long time might be. The Tokio
folks wrote the following about it a few years
ago</a>:</p>}
^{1</sup>
Modern storage and networking devices can get pretty darn fast.
A modern PCIe connection can transfer 128GB/s or 1Tb/s. Sub-millisecond response
times for storage are not unheard of either. And even network calls, for example
between processes (containers) or between machines in a data center, can resolve
incredibly quickly.</p>
</div>}
[…] it is very hard to define "progress". A naive definition of progress is
whether or not a task has been scheduled for over some unit of time. For
example, if a worker has been stuck scheduling the same task for more than
100ms, then that worker is flagged as blocked and a new thread is spawned. In
this definition, how does one detect scenarios where spawning a new thread
reduces throughput? This can happen when the scheduler is generally under load
and adding threads would make the situation much worse.</p>
</blockquote>
Even if we decide something is "blocking" after an arbitrary cutoff - that might
still not do us any good if the overall throughput of the system is reduced.
Determining whether something is objectively blocking or not might be the most useful
question to be asking. Alice Ryhl (also of Tokio) had the following to
say</a>:</p>

In case you forgot, here's the main thing you need to remember:
Async code should never spend a long time without reaching an .await</code>.</p>
</blockquote>
I believe this is a much more useful framing. Rather than attempting (and
failing) to categorize calls as "blocking", we should identify where we're
spending longer stretches of time without yielding back to the runtime - and
ensure we can correct it when we don't. In async-std I introduced the
task::yield_now</code></a>
function to help with this 2</a></sup>. Tokio subsequently adopted
it</a> 3</a></sup>, and
for WASI's async model a similar function is being considered too.</p>
^2</sup>The original issue where I introduced this idea is still
available</a> in case anyone is
reading about the motivation for this at the time. We very clearly modeled it
after thread::yield_now</code>, but async.</p>
</div>
^{3</sup>
I believe they've been working on an alternative system to this
for a few years now, but I'm not sure what the state of that is.</p>
</div>
The goal of this post is to show by example the ambiguity present when
attempting to create an objective classification of which code counts as
"blocking". As well as taking the opportunity to highlight previous writing on
the topic.</p>}Special thanks to Jim Blandy</a>, who during my
review of The Crab
Book</a>
convinced me it is actually fine to use a synchronous mutex in asynchronous
code, in turn pointing me towards Alice's writing</a>.</em></p>

Reframing WIT as primarily a machine format
Sun, 19 Nov 2023 00:00:00 +0000
This is a "short" post. Unlike my usual (lengthy) research pieces, this is
mostly a train-of-thought, low-edit post intended to convey an idea or
perspective I thought would be useful to write down.</em></p>
edit(2023-11-19):</strong> I've changed the title of this post from "Rethinking WIT"
to "Reframing WIT". I strongly hold that WIT being human-readable is a good
thing we shouldn't change, even though in this post I make the case that despite
that it probably shouldn't be the primary way we interact with it.</em></p>

Earlier this year I wrote about how I believe HTML is too complex to be
feasibly written by hand</a>, and
rather than linting manually authored HTML for errors it's probably more
effective to treat HTML as a target which should be compiled to. I ended up
putting these ideas to practice in the html crate for
Rust</a>, and I think that ended up being
roughly the right idea. At the core of this was a reframing of "HTML" as this
thing that's meant to be read and written by humans, to a format that is
primarily intended to be produced and ingested by machines.</p>
Recently I've been using WIT - the IDL format used to define Wasm
Components with - and I'm slowly starting to form opinions about it. Yesterday I
wrote about why I believe ABIs can be described as: "A type system + object
encoding + calling
convention"</a>. But today I
want to take a slightly different perspective on WIT and share an opinion I've
started forming: I believe that WIT should primarily be produced and consumed by
tools, and not by humans. This is a departure from how people have been
reasoning about WIT so far - which is to think of WIT as a human-readable format
people will want to read and write by hand. In this post I want to explain why I
believe this is probably a better way to reason about WIT.</p>
Interacting with WIT</h2>
I believe that from a programmer's perspective all WIT usage can be roughly
broken up into two categories: consuming WIT definitions to use in programs. And
producing WIT definitions to be used in other programs, or to run your program
in a certain runtime. I believe that when we import</em> WIT, we probably don't
want to read WIT files directly. What we're more likely interested in are the
bindings which would be generated for our programming language. Or perhaps if
we're familiar with WIT itself, we might be interested in an interactive doc
explorer a la rustdoc</code> to navigate the type hierarchy.</p>
If we're going to be exporting code using WIT, then we probably want to start by
writing the code itself. In Rust when we want to serialize types we don't
typically start writing a JSON Schema</a> - instead we
start by writing our structs and adding a serde
derive</a> at the top. And we don't just do this for JSON, we take a similar
data-first approach</a> for command
lines too. I believe Wasm Components and their WIT definitions would likely do
well with a similar approach. Because integrating it into the language like this
is by far the path of least resistance.</p>
I believe this will probably hold true for the majority of programmers. Rather
than writing or consuming types authored in WIT, it'll be much easier to reason
about components in terms of how they integrate into the language. And if for
some reason reasoning about the bindings is not sufficient, a documentation tool
will probably be the easier to way to explore WIT APIs directly.</p>
Why perspectives may differ</h2>
WASI Preview 2 hasn't been released yet, and so up until this point the majority
of people working with WIT haven't actually been programmers trying to build
things with</em> WIT. So far the people using WIT have mostly been the people
working on WASI itself, primarily working to define the language-agnostic
interfaces everyone will be writing their programs against.</p>
And so I think this is a likely explainer for why the readability of WIT has
been highly valued so far. The work the majority of people using WIT today is to
write WIT by hand.  But I believe that once Preview 2 releases, this will likely
become a minority use case. If done right, most people working with WIT will be
using it via programming languages instead.</p>
Even for system rewrites I believe it's likely easiest to generate bindings for
an existing impl directly from a language. Then take the WIT definitions
generated by that, and use that as the interface for the new implementation. At
no point does this require people to directly interact with WIT definitions.</p>
On tradeoffs</h2>
To me it makes sense that if we believe that the primary way people use WIT is
either write it by hand, or read the source files, that the property of
"readability" is one we're least keen to compromise on. I'm not quite sure about
the exact tradeoffs; but my understanding is that the "readability" of WIT has
been fairly important.</p>
So far I've been making the case that I believe people are more likely to be
ingesting WIT using language-specific bindings, producing it via language
ascriptions, and reading it via documentation tools. And so if that's the case,
then "readability" may perhaps be something we're more willing to compromise on
in favor of other properties.</p>
That's not that I'm trying to argue that being able to write WIT by hand
or read it directly isn't important. What I'm trying to say is that I don't
believe it necessarily is the most</em> important property.  There may be other
properties, like expressivity or machine-parsability, which would compromise
readability. If we value "readability" differently, then we may very well end up
making different decisions.</p>
Where to go from here?</h2>
I believe that the more closely we can integrate Wasm Components/WASI with
language toolchains and guest languages, the easier it will be for people to
adopt it. The explainer I've used for this in the past is in terms of "deltas".
The less people need to deviate from their existing workflows to do a new thing,
the more likely it is they'll want to try the new thing. Applied to Components: the
more it feels like importing and exporting Components is like using any other
types in that language, the more likely it is that people are willing to use it.</p>
With that in mind, I would like us to start thinking about ways in which we can
remove manually authored WIT entirely from the end-to-end experience. If done
right, both importing and exporting WIT definitions from guest languages should
feel completely native. As if we were importing or exporting any other
library written in that same language. I feel like we're close to that for some
languages 1</a></sup>, but I believe we can probably improve on it.</p>
^{1</sup>
I want to add here: the existing efforts I've seen on this front
are really impressive! I'm in no way trying to snub the existing work done!</p>
</div>}The other thing I think we should explore is a native wit-doc</code> tool which can
be used to render WIT definitions to HTML. My thinking is that this should be
modeled loosely after rustdoc or Swift's DocC. Given plans are plans to make
progress on the Warg registry next year - this seems like something which could
integrate nicely with that too. Over in Rust land, one of my least favorite
splits is between "docs.rs" and "cargo.io" - in my opinion it would make sense
to merge these two. And given wit-bindgen already runs in the
browser</a>, we could probably
also generate previews of what generated output bindings for various programming
languages would look like. Not unlike the SDK example generators used in some of
the world's fanciest docs</a>.</p>
Conclusion</h2>
In this (short) post I've tried to make the case that while historically WIT has
been authored by hand and read from source files, that may not necessarily be
continue being true in the future. To me it seems more likely that once Preview
2 is released, people using Wasm Components will primarily be interested in
using Wasm Components directly from source languages. Since that is the path of
least friction. And in the case people need to look up WIT APIs directly, a
(hosted) documentation tool will probably provide a better experience than
reading files directly from source.</p>
I argue that if we no longer believe that people will primarily interact with
WIT by manually authoring WIT files, and reading them from source, that the
"readability" property of the WIT format may become less important. That's not
to say it should ever be deemed unimportant, since there are valid reasons to
interact directly with WIT files. But it may be the case that if change how we
value this property, we will become more willing to trade it off in favor of
other properties.</p>

Reasoning about ABIs
Sat, 18 Nov 2023 00:00:00 +0000
Note: this post is not my usual research posts; but more a loose collection of
thoughts I've been thinking of recently. I might start tagging these as "short",
since I'm writing and publishing before fully validating.</em></p>
Not long after I started programming I developed an intuition for what APIs are:
they are the interfaces we use in our applications to communicate between
distinct components. But there was this other term, "ABI", which I didn't quite
understand. "Application Binary Interface" is not very descriptive if you
haven't used it before. Does it mean "binary" because it's encoded? But APIs use
encodings too. If an ABI is separate from an API, are we not supposed to program
against ABIs? What are the differences?</p>
Over the past few months I've been helping out with getting WASI Preview 2 over
the finish line1</a></sup>. Part of WASI Preview 2 are Wasm Components and WIT
definitions, which both are part of an overarching system called "The
Component Model". In broad terms, Components in Wasm are able to communicate
with one another via a stable ABI encoding, which is accompanied by WIT (an
Interface Definition Language) that can be used to describe system interfaces.
For example like so:</p>
^{1</sup>
ETA for WASI Preview 2 at the time of writing: either December this year or
January next year; it's November now.</p>
</div>}package example:cat;
</span>
</span>interface cat {
</span>  meow: func() -> string;
</span>}
</span>
</span>world cats {
</span>  export cat;
</span>}
</span></code></pre>
This defines a package cat</code> in the namespace example</code>, providing a set of
export cats</code>, which includes the cat</code> interface. In the component model we
know how to take this interface, and encode this as a Wasm Component. Which can
then be imported directly by other components, and have bindings generated for
it for any language via
wit-bindgen</code></a>. WASI
(WebAssembly System/Standard Interfaces) are all defined in terms of WIT, and
describe standard ways of interfacing with things like system clocks,
filesystems, networks, and in the future caches and queues too.</p>
I'm really used to thinking of ABIs in terms of data encodings and byte
offsets. In Rust we can slap pub extern "C"</code> on any type to change the way it's
laid out in memory. But it has limitations; like for example it doesn't know
what to do with async fn</code>.  In the past I've also worked with other IDL formats
such as WebIDL</a>,
COM</a>, and
WinMD</a>. And while
all of these have helped me understood what an ABI does</em>, they haven't really
helped me understand what an ABI fundamentally is</em>. But now that I've worked
with WIT, I now believe that the better way to think of ABIs as just a different
application of type systems:</p>

Application Binary Interface</strong>: A type system with a specified serialization
format and calling convention 2</a></sup>.</li>
Programming Language</strong>: A type system with runtime behavior (operational
semantics</a>).</li>
</ul>
^{2</sup>
A "calling convention" determines the way functions are
invoked; how arguments are mapped to registers, etc.</p>
</div>}The Wasm Component model doesn't just define types like "string" or "record",
it also has a (limited 3</a></sup>) notion of generic types too. For example result</code> and
option</code> are generic types, so you can define functions which return a
result<string, error></code> to indicate they're fallible. I never considered that
ABIs could contain ADTs 4</a></sup>, but WIT clearly has it. Which to me signals that (with
some constraints), we can apply the wealth of type system theory we've built up
over the years to ABIs as well.</p>
^{3</sup>
There are currently four built-in generics in WIT. It's unclear
whether it'll support user-defined generics in the future since that would
complicate implementations. I'd love to see it though.</p>
</div>}^{4</sup>
"Algebraic Data Type" - which are types which can carry data. These
include both enums (sum types) and structs (product types) in Rust. What's cool
about them is that they're compositional because an ADT can carry more ADTs.</p>
</div>}The boundaries between ABIs and programming languages in practice can often get
murky though. As we've seen with extern "C"</code> in Rust, programming languages can
define ABIs internally as well. And vice versa: some ABIs may depend on runtime
semantics of a language too. But in its broadest terms: I believe an ABI must</em>
define a data encoding, but doesn't need to define runtime semantics. And a
programming language must</em> define operational semantics, but doesn't have to
define an encoding 5</a></sup>. And I feel like that provides a much crisper way
to think about what ABIs actually are, rather than just what they're used for?</p>
^{5</sup>
A data encoding is still necessary to actually encode instructions
on a machine; but that doesn't need to be part of the programming language - it
can be an implementation of the compiler. And multiple compilers for the same
language can truthfully compile the same language, despite not needing to agree
on the encoding. As long as the operational semantics end up working as specified.</p>
</div>}Two Kinds of ABIs (added 2023-11-18)</h2>
I think it's fair to say there are two ways to think about ABIs. One is just as
the low-level encoding of types + calling conventions. The WASI Component model
has a document dedicated to what is called "the canonical
ABI"</a>
which only specifies the encoding and calling conventions.</p>
The way I'm talking about ABIs here in this post is one step above that, where
we not only consider the WASI canonical ABI, but also the WIT IDL format as a
part of "ABI". Both are designed with each other in mind, and having WIT without
the canonical ABI doesn't make much sense. This feels very similar to my
experience working on windows-rs</a>,
which ingests WinMD definitions and knows how to project those into the WinRT
calling convention. Maybe it makes sense to just have ABIs refer to the
low-level calling convention, and have a different name for the IDL + ABI
system? But given how tightly these are linked, I'm not sure it's worth
distinguishing between the two? Or whether talking about it in terms of like:
"ABI encoding" and "ABI description" are enough? If anyone knows of better
terminology for this, I'd love to hear it!</p>
Why ABIs require a type system (added 2023-11-18)</h2>
Jonathan Pallant</a> asked some really good
questions on Mastodon about whether high-level IDLs such as WIT can conceptually
be separated from the underlying encoding. From their perspective as an embedded
systems engineer, ABIs are about object encodings + calling conventions. Since
from their perspective it was primarily about encodings, the need for a
higher-level IDL or a type system didn't really make sense. Which is a really
fair perspective, and it made me scratch my head a little. But I think I've
found an explanation for why the two can't be separated.</p>
Take for example C. There is the programming language "C", and then there are
separate calling conventions. For example on POSIX x86-64 platforms it uses the
"x86-64 System V" calling convention. On 32-bit POSIX it uses the 32-bit "System
V" calling convention. Gankra has written a great post about the C language and
the importance of ABIs</a>.</p>
I believe that when we talk about "C ABIs" (plural as there is no canonical
ABI), it's not enough to just point at the encoding and calling conventions. As
Gankra covered in her post; it's also important that there is a shared
understanding of types. You can't implement the "C ABI" if you don't also encode
what an int</code> in C is. If intmax_t</code> was not part of the "C ABI", then it would
not be an issue to change it either.</p>
Taking this back to Wasm Components: while there is something called the
"Canonical
ABI"</a>
which defines the encoding of Wasm Components. If anything wants to implement
the "Wasm Component ABI", they can't implement the encodings provided by that.
WASI defines a number of built-ins which have a stable encoding, and thus can be
considered part of the ABI too.</p>
This to me feels like the most convincing argument for why when we talk about
ABIs we cannot meaningfully separate the higher-level language (C or WIT) from
the lower-level encoding ("x86-64 System V" or "WASI Canonical ABI"). The types
defined in the higher-level language are a part of the overall contract we call
"ABI" 6</a></sup>. And those types cannot be defined without also defining a
type system to define those types in.</p>
^6</sup>The abi-cafe project</a> is a
good example here. It tests the compatibility of various projects which output C
ABI. And one of its trophy cases is an incompatibility between x86 linux clang,
and gcc on how __int128</code> should be encoded when passed on-stack. And that can
only be an issue if __int128</code> is considered part of the C ABI.</p>
</div>

Iterator as an Alias
Wed, 08 Nov 2023 00:00:00 +0000
This is another short post covering another short idea: I want to explain the
mechanics required to make Iterator</code> an alias for the Coroutine</code> trait. Making
that an alias is something Eric Holk brought up yesterday. We then talked it
through and mutually decided it probably wasn't practical. But I thought about
it some more today, and I might have just figured out a way we could make work?
In this post I want to briefly sketch what that could look like.</p>
⚠️ DISCLAIMER: I'm making up a ridiculous amount of syntax in this post. None
of this is meant to be interpreted as a concrete proposal. This is just me
riffing on an idea; which is a very different flavor from some of my multi-month
research posts. Please treat this post for what it is: a way to share
potentially interesting ideas in the open. ⚠️</em></p>
Iterators vs Coroutines</h2>
An iterator is a type which has a next</code> method, which returns an Option</code>.
Every time you call next</code> it either returns Some(T)</code> with a value, or None</code>
to indicate it has no data to return. A coroutine is a variation of this same
idea, but fundamentally it differs in a few ways:</p>

Coroutines can take arguments to their resume function. Which allows yield</code> statements to evaluate to values.</li>
Coroutines can return a different type than they yield. Iterators can only yield a type, but will always return ()</code>.</li>
</ul>
Just so we're able to reference it throughout this post, here are the
(simplified) signatures:</p>
// The iterator trait
</span>trait </span>Iterator {
</span>    </span>type </span>Item;
</span>    </span>fn </span>next</span>(&</span>mut </span>self</span>) -> Option<</span>Self::</span>Item>;
</span>}
</span>
</span>// The coroutine trait
</span>trait </span>Coroutine<R> {
</span>    </span>type </span>Yield;
</span>    </span>type </span>Return;
</span>    </span>fn </span>resume</span>(&</span>mut </span>self</span>, </span>arg</span>: R) -> CoroutineState<</span>Self::</span>Yield, </span>Self::</span>Return>;
</span>}
</span>
</span>// The enum returned by the coroutine
</span>enum </span>CoroutineState<Y, R> {
</span>    Yielded(Y),
</span>    Complete(R),
</span>}
</span></code></pre>
Because it's useful to show, here's roughly what (I imagine) it could look like
to implement both iterators and coroutines using generator function notation:</p>
// Returning an iterator
</span>// (syntax example, entirely made up, not an actual proposal)
</span>gen </span>fn </span>eat_food</span>(</span>cat</span>: &</span>mut</span> Cat) -> yields Purr {
</span>//                            ^          ^
</span>//                            |          |
</span>//                            |  The type yielded (output)
</span>//                            |
</span>//                      The return type is always `()`
</span>//  
</span>    </span>for </span>0</span>..</span>10 </span>{
</span>        </span>// yield a `Purr` ten times
</span>        </span>yield</span> Purr::new();
</span>    }
</span>}
</span>
</span>// Returning a co-routine
</span>// (syntax example; entirely made up, not an actual proposal)
</span>gen</span>(Food) </span>fn </span>eat_food</span>(</span>cat</span>: &</span>mut</span> Cat) -> Nap yields Purr {
</span>// ^                                   ^          ^
</span>// |                                   |          |
</span>// |                                   |  The type yielded (output)
</span>// |                                   |
</span>// |                           The return type (output)
</span>// |
</span>// The type `yield` resumes with (input)
</span>    </span>for </span>0</span>..</span>10 </span>{
</span>        </span>// Every time we yield a `Purr`, yield resumes with `Food`.
</span>        </span>let</span> food = </span>yield</span> Purr::new();
</span>        cat.</span>eat</span>(food);
</span>    }
</span>    Nap::new() </span>// We end the function by returning a `Nap`
</span>}
</span></code></pre>
There are probably better ways of writing this. For example: imo the types
passed to yield</code> should be a special entry in the arguments list. We can
possibly drop the -></code> requirement for iterator-returning generator functions.
And using gen</code> as a prefix probably isn't necessary either. But that's all
stuff we can figure out later. I at least wanted to make sure folks had had an
opportunity to see what functionality coroutines provide when writing generator
functions.</p>
Iterators as an alias for Coroutines</h2>
Practically speaking this means that co-routines are a superset of functionality
of iterators. Enough so that if we squint we might think see it work out. We can
recreate the iterator trait if we implement a coroutine with this signature:</p>
impl </span>Coroutine<()> </span>for </span>MyIterator {
</span>    </span>type </span>Yield = SomeType;
</span>    </span>type </span>Return = ();
</span>    </span>fn </span>resume</span>(&</span>mut </span>self</span>, </span>arg</span>: ()) -> CoroutineState<</span>Self::</span>Yield, ()> { .. }
</span>}
</span></code></pre>
It's tempting to think we could create a trait alias for this. But we run into
trouble here: the coroutine trait wants to be able to return a CoroutineState</code>.
While Iterator</code> is hard-coded to return Option</code>. And we can't change the existing Iterator</code> trait to take anything but an option. So what should we do here.</p>
Fallible functions</h2>
We can't change Iterator</code>, but we can</em> change Coroutine</code>. What if rather than
hard-coding that it returned a CoroutineState</code>, we instead said that it could
return anything which implements the Try</code> trait? Try</code> is implemented for both
not only Option</code>, but also CoroutineState</code>! And if we had try</code> or throws</code>
functions, we could make that configurable like so:</p>
// The coroutine trait
</span>trait </span>Coroutine<R> {
</span>    </span>type </span>Yield;
</span>    try </span>fn </span>resume</span>(&</span>mut </span>self</span>, </span>arg</span>: R) -> </span>Self::</span>Yield throws { .. }
</span>}
</span></code></pre>
edit(2023-12-01)</strong>: Yosh from the future here. Scott McMurray helpfully
provided this desugaring of what an un-handwaved desugaring of this signature
could look like. Crucially it shows that we could have a fully generic throws</code>
clause without defining what exactly it throws:</p>
// The coroutine trait
</span>trait </span>Coroutine<R> {
</span>    </span>type </span>Yield;
</span>    </span>fn </span>resume</span>(&</span>mut </span>self</span>, </span>arg</span>: R) -> <</span>Self::</span>Residual as ops::Residual<</span>Self::</span>Yield>>::TryType { .. }
</span>}
</span></code></pre>
I'm super hand-waving things away here</del>. But roughly what I'm meaning to describe
here is that this can return any type implementing Try</code> which returns
Self::Yield</code>, without describing the specific Try::Residual</code> type. Meaning
that could equally be an Option</code>, a CoroutineState</code>, or a Result</code>. The next
function should not distinguish between those. If we take that path, then we
could use that to rewrite the Iterator</code> trait as an alias for Coroutine</code>. We
don't have a syntax for this, so I'll just make up a syntax based on return
type notation</a>:</p>
// A trait alias, hard-coding both the return type and resume argument to `()`.
</span>trait </span>Iterator = Coroutine<(), next(..) -> Self::Yield throws None>;
</span></code></pre>
There's probably a better way of writing this. But syntax is something we can
figure out. Semantically this would make the two equivalent, and it should be
possible to write a way to make these work the same way.</p>
Also on a quick syntactical note: the throws None</code> syntax here would operate
from the assumption that the right syntax to declare which type of residual
you're targeting is by writing a refinement. So to use Option<T></code> as the Try</code>
type, you'd write throws None</code>. For Result<T, E></code> you'd write throws Err(E)</code>.
And so on. That way if you strip the throws Ty</code> notation, you'd be left with
the base function signature. In a fun mirrored sort of way, I don't think we'd
need to prefix fallible functions with a try fn</code> notation. Just writing
throws</code> in the return type should be enough. I think the same should be true
for generator functions too; just writing yields T</code> in the signature should be
enough.</p>
Support for pin</h2>
Now, we've gone through an entire post to show how we could turn Iterator</code> into
an alias for Coroutine</code>. But there is one more detail we've skipped over: in
the compiler the Coroutine</code> trait is used to implement future state machines
with, created using the async</code> keyword. In order for borrows to work across
.await</code> points in async functions, the Coroutine</code> trait needs to support
self-references. Today that's done using the Pin</code> self-type, which means the
actual signature of Coroutine</code> today is:</p>
trait </span>Coroutine<R> {
</span>    </span>type </span>Yield;
</span>    </span>type </span>Return;
</span>    </span>fn </span>resume</span>(</span>self</span>: Pin<&</span>mut Self</span>>, </span>arg</span>: R) -> CoroutineState<</span>Self::</span>Yield, </span>Self::</span>Return>;
</span>    </span>//        ^ this changed
</span>}
</span></code></pre>
So Coroutines</code> must be able to express self-references. And iterators currently
don't support that. Which means we need to find a way to bridge the two. Luckily
this is something we might be able to resolve if we consider pinning as an
orthogonal capability which we can express as a trait
transformer</a>.
Ideally what we'd do is make Coroutine</code> and by extension Iterator</code> generic
over: "may be self-referential". I talk about this more in my latest post on the
async
trait</a>;
specifically pointing out that async iterators don't need to be self-referential -
and generator functions probably do want to be able to have self-references.
Making Iterator</code> an alias for Coroutine</code> further reinforces that requirement.</p>
Conclusion</h2>
This was a quick post to illustrate a way we could make Iterator</code> a more
narrowly scoped alias for Coroutine</code>. I'm not sure we should do this; from my
perspective we have a few options:</p>

Never stabilize the co-routine trait. This is probably fine, since being able
to pass arguments to yield</code> or change the return type from the yield type don't
seem like major features. Useful, but not critical</em>.</li>
Implement Coroutine</code> as a superset of Iterator</code>. Keeping both, but using
blanket impls</a> make both work
with one another. It's feasible, but it feels like we're creating "V2"
interfaces which aren't interchangeable. You could pass an iterator everywhere
a generator was accepted. But you couldn't pass a generator everywhere an
iterator was accepted - unless we carve out a special exception just for this,
which seems like a pretty bad hack.</li>
Make Iterator</code> an alias for Coroutine</code>. That's what this post is about. It
would be the way we bring some of the core unstable functionality out of the
stdlib and make it available to users. Without creating an "iterator v2"
trait. If we're serious about ever wanting to stabilize Coroutines, this seems
preferable.</li>
</ol>
A nice side-effect of this approach is that we could use this as a way to
normalize our syntax as well. Right now we're discussing adding "generator
functions" in the 2024 edition. But we're making a distinction between
"iterators", "generators", and "co-routines". If we were to adopt the aliasing
model, then the iterator trait would be the same as the generator trait. And we
would no longer have a need for another, separate concept for "co-routines".</p>
I for one like the idea that a "generator function" returns an impl Generator</code>.
And even if I don't think that working on that should be a priority, I like that
we might now have a mechanism to make this a reality by creating a trait alias,
hard-coding certain params, and relying on the Try</code> trait.</p>

What is a team?
Wed, 08 Nov 2023 00:00:00 +0000
update (2023-11-08): I've swapped the usage of "horizontal" and "vertical" in this post based on feedback.</em></p>
This is going to be an ultra-short post to share an idea I've been mulling over
in my head for a while. Yesterday I raised the question again about WG Async
getting checkboxes for approval on decisions. For the past five or so years
we've had a version of the async working group driving the design of async Rust.
With the past three years being a lot more organized, with a steady roster,
leadership, and mandate. WG Async getting the ability to set checkboxes for
approval would mostly just be a formalization of the existing role we play in
the project.</p>
However, in order to give checkboxes to the Async WG there is a question
about the way it should integrate into the project. Historically we've
distinguished between: top-level "teams", and domain-specific "working groups".
Teams are formal, first-class entities with voting rights. While working-groups
are not. This current structure still exists in the project, and so in order for
WG Async to get checkboxes we'd need to become a "team". And with that there is
also a question about whether we should be a sub-team (like T-Types) or a
top-level team.</p>
Top-level teams get a seat on the council. Sub-teams are only represented on the
council by their parent teams. In the case of T-Types the parent teams are
"Lang" and "Compiler". In the case of WG Async the parent teams should be at
least "Lang" and "Libs-API"; but arguably also "Compiler" since we tend to
implement the features we've designed.</p>
And this is where I think the current model runs into trouble. Ideally we'd
think of the organization hierarchy as a tree: there are teams and there are
sub-teams. But because sub-teams can have multiple parents, our tree is really a
DAG</a>. And that becomes pretty complex to reason about. So instead I want to
propose an alternate model to our organizational DAG: I think we might be better
off reasoning in terms of vertical</em> and horizontal</em> teams.</p>

vertical teams</strong>: These are teams such as Lang, Libs, Compiler, and Cargo.
They are their a domain in their own right, and largely focus on one area of work.</li>
horizontal teams</strong>: These are teams such as Types, Async, and possibly also
Moderation. These are teams whose domain spans across multiple vertical teams, but
rather than going broad - they specialize in a specific subject.</li>
</ul>
Categorizing teams like this doesn't directly help answer questions about who
should have a seat on the council, nor things like membership. Those are
questions of policy, and I'm intentionally keeping that out of scope for this
post. But I hope it can help us explain why teams like "Async" feel different
from teams like "Compiler".</p>
Specifically I also believe that horizontal teams can help fill the gaps left
between verticals. The Types Team exists because we identified that it's not
enough to think about the language, or the compiler - but there is a need to
also consider how the two interact. Similarly: WG Async doesn't just consider
the async language, library, or compiler aspects. It needs to consider how the
language features will be implemented, and how that in turn will affect
libraries.</p>
I don't know what we want to name these groups. In my opinion both vertical and
horizontal teams should be teams in their own right; with neither being "more
real" than the other. But as I've said: if we were to adopt that we'd need to
reason about a bunch more things, because the current project structure attaches
certain things to "top-level" teams right now.</p>
There is also a final question about the difference between "teams" and
"initiatives". The way I'm reasoning about those is as follows:</p>

teams</strong>: Permanent in nature. Exist for an unbounded amount of time</em>, cover
a broad area, and have a relatively steady membership.</li>
initiatives</strong>: Ephemeral in nature. Exist for a bounded amount of time</em>,
created for a specific purpose under a team, and spun down once that purpose
has been served.</li>
</ul>
From my perspective the purpose of an initiative is to organize specific sets of
work under a team, without requiring the people doing that work are members of
that team. For example, neither Oli nor I are members of the Lang team, but the
Effect Generics are a language-level effort. The Effects Initiative allows us to
do this work under the purview of the Lang Team without either of us being on</em>
the Lang Team.</p>
And that's about it. I think if we're going to be talking about teams and
sub-teams, I think it's clearer to reason about teams in terms of "vertical
concerns" (fixed domains like "compiler", "language", "stdlib") and "horizontal
concerns" (cross-cutting domains like: "the type system", "async support",
"moderation").</p>

Async Iteration III: The Async Iterator Trait
Tue, 26 Sep 2023 00:00:00 +0000

This post is part of the Async Iteration series:</em></p>

Async Iteration I: Async Iteration Semantics</a></li>
Async Iteration II: The Async Iterator Crate</a></li>
Async Iteration III: The Async Iterator Trait</a> (this post)</li>
</ol>

Async Functions in Traits (AFIT) are in the process of being
stabilized</a> and I figured it
would be a good time to look more closely at the properties they provide. In
this post I want to compare AFIT-based traits with poll-based traits, using the
"async iterator" trait as the driving example. But most everything which applies
to async iterator, will also apply to other traits such as async read and async write.</p>
In this post I will make the case that the best direction for the stdlib is to
base its async traits on AFITs. The intended audience for this post is primarily
my fellow members of WG-Async, as well as members of T-Lang and T-Libs. To read
a summary of the findings jump ahead to the conclusion</a>. This post
assumes readers are familiar with the inner workings of Rust's async systems, as
well as a familiarity of the tradeoffs being discussed.</p>
fn poll_ vs async fn</h2>
To provide some flavor to what I'm talking about, in this post we'll be
discussing the "async iterator" trait, asking the question whether we should
base it on fn poll_next</code> or async fn next</code>. Here are both variants
side-by-side:</p>
// Using `fn poll_next`.
</span>trait </span>AsyncIterator {
</span>    </span>type </span>Item;
</span>    </span>fn </span>poll_next</span>(</span>self</span>: Pin<&</span>mut Self</span>>, </span>cx</span>: &</span>mut </span>Context<'_>) -> Poll<Option<</span>Self::</span>Item>>;
</span>}
</span></code></pre>
// Using `async fn next`.
</span>trait </span>AsyncIterator {
</span>    </span>type </span>Item;
</span>    async </span>fn </span>next</span>(&</span>mut </span>self</span>) -> Option<</span>Self::</span>Item>;
</span>}
</span></code></pre>
I expect pretty much everyone will agree that on a first look the async fn next</code>-based trait seems easier to use. Rather than needing to think about what
Pin</code> is, or how Poll</code> works, we can just write our async functions the way we
usually do, and it will just work</em>. Pretty neat!</p>
But that's just on the surface. Does that still hold if we look more closely?
Concerns have been raised about the performance of async fn next</code>, claiming not
only would it perform less well. It's also alleged that async fn next</code> does not
provide essential features, even going so far to claim that fn poll_next</code> is
fundamentally lower-level and thus the only reasonable choice for a systems
programming language. In the remainder of this post we'll be going over those
claims, and show why upon closer examination they do not appear to hold.</p>
Performance</h2>
Let's start with the most obvious one: performance. At its core Rust is a
systems programming language, and in order to properly cater to its niche it
tends to only provide abstractions which have comparable performance to their
hand-rolled versions. The claim is that poll_next</code> should provide better
performance than async fn next</code> since we're compiling it by hand. But when
actually measured, the two approaches appear to compile to identical assembly in
various configurations - meaning they will have identical performance.</p>
But don't just take my word for it, we can use examples to substantiate this.
Let's create a simple "once" future which holds some data, and when polled it
will return that data. Rather than using complex async/.await machinery, we'll
be creating a new function poll_once</code> which constructs a dummy waker in-line
and can be used to poll a future exactly once:</p>
pub fn </span>call_once</span>() -> Poll<Option<</span>usize</span>>> {
</span>    </span>let mut</span> iter = </span>once</span>(</span>12</span>usize</span>);
</span>    </span>poll_once</span>(iter.</span>next</span>())
</span>}
</span></code></pre>
Let's start by evaluating what this looks like when implemented using fn poll_next</code>. We could write this as follows:</p>
struct </span>Once<T>(Option<T>);
</span>impl</span><T> AsyncIterator </span>for </span>Once<T> {
</span>    </span>type </span>Item = T;
</span>    </span>fn </span>poll_next</span>(</span>self</span>: Pin<&</span>mut Self</span>>, </span>_cx</span>: &</span>mut </span>Context<'_>) -> Poll<Option<</span>Self::</span>Item>> {
</span>        </span>// SAFETY: we're projecting into an unpinned field
</span>        </span>let</span> this = </span>unsafe </span>{ Pin::into_inner_unchecked(</span>self</span>) };
</span>        Poll::Ready((&</span>mut</span> this.value).</span>take</span>())
</span>    }
</span>}
</span></code></pre>
When polled we project Self</code> into its fields, which is just the Option</code> type.
We then call .take</code> to extract the value, or panic if there is none. This
should be fairly straight forward. If poll_once</code> creates a non-atomic dummy
waker (this is just the first example), the compiler will compile this code down
to the following x86 assembly (compiler explorer</a>):</p>
example::call_once:
</span>        </span>mov     </span>eax</span>, </span>1
</span>        </span>mov     </span>edx</span>, </span>12
</span>        </span>ret
</span></code></pre>
This assembly basically means: "Hey I've got the constant '12' over here -
please move it into the return registry and then exit the function"</em>. That's
about the smallest this function can be without being inlined. Now let's see
what happens if we implement this code using async fn next</code>.  Instead of fn poll_next</code> we can use an async function directly:</p>
pub struct </span>Once<T>(Option<T>);
</span>impl</span><T> AsyncIterator </span>for </span>Once<T> {
</span>    </span>type </span>Item = T;
</span>    async </span>fn </span>next</span>(&</span>mut </span>self</span>) -> Option<T> {
</span>        </span>self</span>.</span>0.</span>take</span>()
</span>    }
</span>}
</span></code></pre>
It's nice we don't have to perform pin projections anymore (more on that later).
But what's the performance like? Well, if this was slower we'd expect it to
generate more assembly. So let's take a look (compiler explorer</a>):</p>
example::call_once:
</span>        </span>mov     </span>eax</span>, </span>1
</span>        </span>mov     </span>edx</span>, </span>12
</span>        </span>ret
</span></code></pre>
The assembly is identical! Why is that? Well, for one: the Rust compiler is
pretty good at generating fast code. But we're also in a bit of a simplified
environment. So far in  our examples we've not using "real" thread-safe wakers,
instead basing our wakers on Rc</code>. What happens if we switch to Arc</code>-based
wakers? Here's the link to a compiler
explorer</a> comparing the two. It now generates a
lot more assembly than before (yay atomics), but luckily we can use diff(1)</code> to
compare the output:</p>
example::call_once:
</span>        ; 21 lines of assembly + calls to another 118 lines
</span></code></pre>
yosh@MacBook-Pro</span> scratch % pbpaste > one.rs
</span>yosh@MacBook-Pro</span> scratch % pbpaste > two.rs
</span>yosh@MacBook-Pro</span> scratch % diff one.rs two.rs
</span>yosh@MacBook-Pro</span> scratch %
</span></code></pre>
The diff</code> output is empty, meaning there are no differences even if we Arc</code>s
to correctly construct our wakers, it just generates a lot more code. But okay
fine, maybe there are more differences? After all: fn poll_next</code> has access to
the Waker</code> and can return Poll</code>, meaning it has low-level control over the
future state machine while async fn next</code> does not. What happens if we want to
provide low-level control over the future state machine from async fn next</code>?</p>
Luckily we've stabilized a simple mechanism for this already:
std::future::poll_fn</code></a>.
This function provides the ability to access the low-level internals of any</em>
future, including AFITs. Let's lower our example to make use of this, shall we?</p>
pub struct </span>Once<T>(Option<T>);
</span>impl</span><T> AsyncIterator </span>for </span>Once<T> {
</span>    </span>type </span>Item = T;
</span>    async </span>fn </span>next</span>(&</span>mut </span>self</span>) -> Option<T> {
</span>        future::poll_fn(|</span>_cx</span>| </span>/* -> Poll<Option<T>> */ </span>{
</span>            </span>// We have access to `cx` here which contains the `Waker`.
</span>            Poll::Ready((&</span>mut </span>self</span>.value).</span>take</span>())
</span>        }).await
</span>    }
</span>}
</span></code></pre>
This seems simple enough: whenever we want to do anything low-level inside of an
async fn</code>, we can use poll_fn</code> to drop into the future state machine. This
should work not just for the async version of the iterator trait, but for all</em>
async traits. There is more to be said about how this interacts with pinning and
self-referential types, but we'll cover that in more detail later on in the
post. To close this out though: what does this compile
to if we call it using "real" wakers? (compiler explorer</a>):</p>
example::call_once:
</span>        ; 21 lines of assembly + calls to another 118 lines
</span></code></pre>
yosh@MacBook-Pro</span> scratch % pbpaste > two.rs
</span>yosh@MacBook-Pro</span> scratch % diff one.rs two.rs
</span>yosh@MacBook-Pro</span> scratch %
</span></code></pre>
That's right: the output remains the same. This gives us a pretty good clue
about what is happening here. Inside the compiler async fn next</code> is desugared
to a future, just like fn poll_next</code> is. And because of basic inlining and
const-folding optimizations, the resulting state machines are identical - which
means that the resulting assembly is identical too. This is exactly how
zero-cost abstractions are supposed to work, and is the entire premise of Rust's
async system</a>. If we ever find
a case where the optimizer doesn't perform those basic optimizations we can then
treat that as a bug in the compiler - not a limitation of the design.</p>
Self-Referential Iterators</h2>
When people say that "async iterator is not the async version of iterator"</em>
they are correct. Well, sort of. If we look at existing</em>
implementations</a>
that is true: it doesn't quite work like the async version of iterator. Instead
what it really is is the async version of "pinned iterator". Which is not a
trait we currently have, but there certainly is a case to be made for it.
Instead it's better to ask whether async iterator should</em> be the "async version
of iterator" - and I certainly believe it should be 1</a></sup>.</p>
^1</sup>Incidentally that has also been the framing of the trait
WG-async has been communicating to T-lang and
T-libs</a>, who have signed off on it.
I'm not suggesting that this decision should bind</em> us (I don't like to rules
lawyer). What I'm instead trying to show with this is that this has been an
accepted framing of what the design should achieve for years now, and we've
already rejected the framing that "async iterator" (or "stream") should be its
own special thing. That certainly can be changed again, but it is not a novel
insight by any stretch.</p>
</div>
Let me explain what I mean by this using examples. In Rust the base iterator API
has an associated type Item</code>, a function next</code> which takes a mutable reference
to self</code>, and returns an Option<Self::Item></code>:</p>
trait </span>Iterator {
</span>    </span>type </span>Item;
</span>    </span>fn </span>next</span>(&</span>mut </span>self</span>) -> Option<</span>Self::</span>Item>;
</span>}
</span></code></pre>
If we did a direct translation to async Rust, we'd have an API which instead of
exposing an fn next</code> exposed an async fn next</code>. The only real difference here
is the addition of the async</code> keyword:</p>
trait </span>AsyncIterator {
</span>    </span>type </span>Item;
</span>    async </span>fn </span>next</span>(&</span>mut </span>self</span>) -> Option<</span>Self::</span>Item>;
</span>}
</span></code></pre>
However, when we look at the ecosystem Stream</code>
trait</a>,
or the currently unstable AsyncIterator</code>
API</a> they are
not implemented in terms of async fn next</code>. Instead they provide an fn poll_next</code> which takes both a pinned reference to self</code>, a mutable reference
to the waker context, and wrap the return type in Poll</code>:</p>
trait </span>AsyncIterator {
</span>    </span>type </span>Item;
</span>    </span>fn </span>poll_next</span>(</span>self</span>: Pin<&</span>mut Self</span>>, </span>cx</span>: &</span>mut </span>Context<'_>) -> Poll<Option<</span>Self::</span>Item>>;
</span>}
</span></code></pre>
In the previous section we've already discussed how you can get access to the
waker context from inside an async function by using poll_fn</code>. So we can pretty
much ignore the waker context and the Poll</code> in the return type. That leaves the
change in the self</code> type. Our async fn next</code> takes &mut self</code>, while this
variant takes Pin<&mut Self></code>. This isn't necessary for the core functionality
of async iterator, since it is pinning more than needed. So simply put, what
we've just written is in fact the async version of this trait:</p>
trait </span>PinnedIterator {
</span>    </span>type </span>Item;
</span>    </span>fn </span>next</span>(</span>self</span>: Pin<&</span>mut Self</span>>) -> Option<</span>Self::</span>Item>;
</span>}
</span></code></pre>
Here we have a non-async version of iterator which takes self as a pinned
reference. This is useful if you ever need to write an iterator which can
operate on self-referential structs. For example: if we ever start thinking of
stabilizing generator functions, we want them to be able to hold references
across yield</code> points. That will require self-referential iterators.</p>
An important insight of this is that the question of whether iterator should be
pinned is orthogonal to whether it is async. Which is illustrated by the fact
that we can reformulate a "pinned asynchronous iterator" just fine using AFITs:</p>
trait </span>PinnedAsyncIterator {
</span>    </span>type </span>Item;
</span>    async </span>fn </span>next</span>(</span>self</span>: Pin<&</span>mut Self</span>>) -> Option<</span>Self::</span>Item>;
</span>}
</span></code></pre>
This can be combined with the poll_fn</code> function as we showed in the previous
section to recreate the low-level semantics of fn poll_next</code>, providing access
to both a pinned self-type and the future's waker argument. To put it plainly:
"is async" and "is pinned" are orthogonal features, and fn poll_next</code>
needlessly combines both</strong>.</p>
Unpin Bounds</h2>
People occasionally ask me about the Unpin</code> trait when I talk about async
versions of traits. For example if you compare
Iterator::next</code></a>
and
futures::stream::StreamExt::next</code></a>,
you will see that the latter has an extra where Self: Unpin</code> bound.</p>
// `Iterator::next`
</span>fn </span>next</span>(&</span>mut </span>self</span>) -> Option<</span>Self::</span>Item>;
</span>
</span>// `FuturesExt::next`
</span>fn </span>next</span>(&</span>mut </span>self</span>) -> Next<'_, </span>Self</span>>
</span>where
</span>    </span>Self</span>: Unpin; </span>// This is different
</span></code></pre>
This extra Unpin</code> bound is only needed when a trait is implemented in terms of
poll functions - which by design take Pin<&mut Self></code>. And so we need a way to
later on opt out of those bounds. You can see this same mechanism in action with
the other poll-based traits, such as
AsyncWriteExt::write</code></a>
which also has a Self: Unpin</code> bound.</p>
Instead if we recognize that the async counterparts to Rust's core traits don't
actually need to be pinned in order to be implemented, we can drop Pin<&mut Self></code> from the signature. And since our type isn't pinned to begin with,
we no longer have to opt-out of it being pinned via Unpin</code> meaning all the
extra Unpin</code> bounds go away. You can see this in action in the
async-iterator</code>
crate</a>
which provides a diverse range of methods on async iterator, none of which
require additional Unpin</code> bounds to function.</p>
Implementation vs Usage</h2>
To stay on the topic of API-shapes: one major downside of fn poll_next</code> is that
the "method to implement" and "method to call" are different methods. In the
regular iterator trait, there is only one method next</code> which is both
implemented and called. Instead the poll-API is only meant to be implemented</em>,
and in virtually all cases the next</code> function is the one you want to call. This
is a major deviation of how all other traits work in the stdlib today.</p>
This isn't just limited to async Iterator</code> either. Presumably we'd want to
adapt this approach for all</em> traits in the stdlib. That means users of async
Rust would need to think of traits in the stdlib as somehow "different", and
remember that they cannot directly implement the methods they're calling. Among
others, the following APIs would be affected:</p>
poll-based stdlib traits</strong></p>
trait name</th> to be implemented</th> to be called</th> is same?</th></tr></thead>

async Read</strong></td> fn poll_read</code></td> async fn read</code></td> ❌</td></tr>
async Write</strong></td> fn poll_write</code></td> async fn write</code></td> ❌</td></tr>
async BufWrite</strong></td> fn poll_fill_buf</code></td> async fn fill_buf</code></td> ❌</td></tr>
async Seek</strong></td> fn poll_seek</code></td> async fn seek</code></td> ❌</td></tr>
</tbody></table>
Instead, if we base these traits on async fn</code>, the method to implement and the
method to call are identical. And as we've covered earlier, if anyone would want
to manually author a poll</code>-based state machine for any of these traits,
poll_fn</code> provides a uniform way to do so for all</em> async traits:</p>
async-fn based stdlib traits</strong></p>
trait name</th> to be implemented</th> to be called</th> is same?</th></tr></thead>

async Read</strong></td> async fn read</code></td> async fn read</code></td> ✅</td></tr>
async Write</strong></td> async fn write</code></td> async fn write</code></td> ✅</td></tr>
async BufWrite</strong></td> async fn fill_buf</code></td> async fn fill_buf</code></td> ✅</td></tr>
async Seek</strong></td> async fn seek</code></td> async fn seek</code></td> ✅</td></tr>
</tbody></table>
This might seem like a minor point, but we have to consider that every deviation
from existing norms is a point of friction for users. To zoom out slightly: I
don't believe that async Rust inherently needs to be much more difficult than
regular Rust. But the missing language features, combined with subtle differences
like these, eventually add up and create an experience which is sufficiently
different that the resulting system feels like an entirely different language.
When in reality it does not need to be. For good measure here are the existing
non-async stdlib traits:</p>
non-async stdlib traits</strong></p>
trait name</th> to be implemented</th> to be called</th> is same?</th></tr></thead>

Read</strong></td> fn read</code></td> fn read</code></td> ✅</td></tr>
Write</strong></td> fn write</code></td> fn write</code></td> ✅</td></tr>
BufWrite</strong></td> fn fill_buf</code></td> fn fill_buf</code></td> ✅</td></tr>
Seek</strong></td> fn seek</code></td> fn seek</code></td> ✅</td></tr>
</tbody></table>
Object Safety</h2>
So far we've only discussed the implementation side of the traits. However that
isn't the complete story, and we need to consider auto traits and other subtle
semantics too. So let's start looking at those, starting with
object-safety</a>.
Out of the box poll</code>-based traits are dyn-safe. Say we wanted to implement a
poll-based version of async iterator which can produce an infinite number of
meows, we could create a dyn variant like so
(playground</a>):</p>
struct </span>Cat;
</span>impl </span>AsyncIterator </span>for </span>Cat {
</span>    </span>type </span>Item = String;
</span>    </span>fn </span>poll_next</span>(</span>self</span>: Pin<&</span>mut Self</span>>, </span>_cx</span>: &</span>mut </span>Context<'_>) -> Poll<Option<</span>Self::</span>Item>> {
</span>        Poll::Ready(Some("</span>meow</span>".</span>to_string</span>()))
</span>    }
</span>}
</span>
</span>fn </span>dyn_iter</span>() -> Box<dyn AsyncIterator<Item = String>> {
</span>    Box::new(Cat {})
</span>}
</span></code></pre>
This is the same as any other dyn-safe trait, and doesn't requiring any
additional steps. Nice! Now what happens if we try and rewrite it to use async fn next</code>.  Well, we would probably try and write it like so
(playground</a>):</p>
#![</span>feature</span>(async_fn_in_trait)]
</span>
</span>struct </span>Cat;
</span>impl </span>AsyncIterator </span>for </span>Cat {
</span>    </span>type </span>Item = String;
</span>    async </span>fn </span>next</span>(&</span>mut </span>self</span>) -> Option<</span>Self::</span>Item> {
</span>        Some("</span>meow</span>".</span>to_string</span>())
</span>    }
</span>}
</span>
</span>fn </span>dyn_iter</span>() -> Box<dyn AsyncIterator<Item = String>> {
</span>    Box::new(Cat {})
</span>}
</span></code></pre>
However if we now try and compile this code we now get the following error:</p>
error[E0038]: the trait `AsyncIterator` cannot be made into an object
</span>  --> src/lib.rs:16:22
</span>   |
</span>16 | fn dyn_iter() -> Box<dyn AsyncIterator<Item = String>> {
</span>   |                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ `AsyncIterator` cannot be made into an object
</span>   |
</span>note: for a trait to be "object safe" it needs to allow building a vtable to allow the call to be resolvable dynamically; for more information visit <https://doc.rust-lang.org/reference/items/traits.html#object-safety>
</span>  --> src/lib.rs:5:14
</span>   |
</span>3  | trait AsyncIterator {
</span>   |       ------------- this trait cannot be made into an object...
</span>4  |     type Item;
</span>5  |     async fn next(&mut self) -> Option<Self::Item>;
</span>   |              ^^^^ ...because method `next` is `async`
</span>   = help: consider moving `next` to another trait
</span></code></pre>
This would not happen if we were using the async-trait
crate</a>, it only happens if we
use the AFIT language feature. But what exactly is going on here? In order for
async traits to work in dyn contexts, we need to find a place to store them
first. In the async-trait</code> crate this is always a Box</code>, but in stable Rust we
can't just do that because we've pinky-promised not to perform any "hidden"
allocations 2</a></sup>. Solving this is pretty complicated, and will require a
feature like
dyn*</code></a>
to land. With dyn*</code> rather than calling Box::new</code> you'd need to call a "dyn
adapter" instead. For
example</a>:</p>
^{2</sup>
Though in the past we have</em> used allocations in language features as
placeholders: that's how async/.await</code> was originally stabilized in 2019. It
wasn't until a year or so later that support landed for async functions which
didn't box in their desugaring.</p>
</div>}fn </span>dyn_iter</span>() -> Box<dyn AsyncIterator<Item = String>> {
</span>    Boxed::new(Cat {}) </span>// NOTE: using the `Boxed` dyn-adapter, not `Box`.
</span>}
</span></code></pre>
Needing to replace Box::new</code> with Boxed::new</code> is not a major difference. And
it's still unclear what the upper bound on the ergonomics of dyn async traits
are, since work on it has been paused for the past year in favor of stabilizing
AFIT first. But it's going to be pretty confusing if certain async traits
require a Box</code>, but other async traits require a Boxed</code> notation.</p>
I believe the better direction is for all async traits in the stdlib to use the
same mechanism to work with dyn, even if initially it's slightly less convenient
at first. We can then gradually work on improving those ergonomics for all async
traits, both in the stdlib and ecosystem. This ensures consistency and enables
us to design solutions which are shared by the entire async ecosystem and
preventing a possible permanent bifurcation of the async trait space.</p>
"Cancellation Safety"</h2>
Another subtle outcome here is the interaction between async traits and the
property has been historically called: "cancellation safety". I'm putting it in
quotes because the property is not actually about whether it is "safe to cancel"
a future. A few months ago I gave a
talk</a>
about this, which I'll summarize in this section. I'll explain what
"cancellation safety" is, how it currently falls short, and how may be able to
fix those shortcomings. I particularly want to show how we can bring the
"cancellation safety" property into the type system, which could enable async
functions (including AFIT) to automatically provide it.</p>
"Cancellation safety" refers to the ability to drop and recreate futures
without any "side-effects" being observable. It's important to note that the word
"safety" in this context has nothing to do with memory safety: in the absence of
linear types</a>, all types
in Rust must be memory-safe to drop, including futures. The "safety" in
"cancellation safety" refers to logical correctness only. Being able to drop and
recreate futures is the most relevant to the select!
macro</a> which commonly does
that in a loop. But it can also come in useful if you're ever authoring future
state machines by hand.</p>
/// # Cancel safety
</span>///
</span>/// This method is cancel safe. If you use it as the event in a
</span>/// </span>`tokio::select!`</span> statement and some other branch completes
</span>/// first, then it is guaranteed that no data was read.
</span></code></pre>
As I've mentioned, "cancellation safety" as a property currently only exists in
documentation. This means that in order to learn whether you can drop and
recreate futures without any actions you need to consult the documentation for
the method. The difference between async fn next</code> and fn poll_next</code> is that
for the former the property will be documented on the implementation, whereas
for the latter the property will be documented on the trait definition.
That practically means that with fn poll_next</code> you'll be able to remember that
all</em> calls to next</code> are "cancellation-safe". Whereas with async fn next</code>
you'll need to remember that for each implementation (or more likely: remember
which impls aren't "cancellation-safe").</p>
But we should be legitimately asking ourselves whether using documentation for
this is the best we can do. RTFM</a> is not
really how we do things in the Rust project, instead much preferring if the
compiler can tell you when you've messed up, which can then explain what to do
instead. This is possible because of Rust's "type
safety"</a> guarantee, and APIs such as
select!</code> are decidedly not type-safe right now. Which is why even after
successfully compiling code, people regularly find runtime bugs in their
select!</code> statements.</p>
I believe a better direction would be to bring "cancellation safety" out of the
docs and into the type system. We could do this by introducing a new
subtrait</a>,
which I'll refer to in this post as AtomicFuture</code> (but we can call it anything
we like really, the semantics are what I care about most). We already have some
precedent for this in the iterator family of traits, where for example the
unstable
TrustedLen</code></a> trait
provides additional guarantees on top of the existing Iterator</code> trait. I
imagine this would look something like this:</p>
//! ⚠️  This uses a placeholder name and is intended as a design-sketch only. ⚠️
</span>
</span>/// A future which performs at most a single async operation.
</span>trait </span>AtomicFuture: Future {}
</span></code></pre>
One of the downsides of the current "cancellation safety" system is that we have
to manually inspect anonymous futures whether they're cancellation-safe or not.
I believe by bringing "cancellation safety" into the type system the compiler
should be able to figure it out automatically if the following cases are met:</p>

No local state:</strong> The async context must contain at most one .await</code>
point. That way if a future is cancelled it will never occur between two
.await</code> points.</li>
Virality:</strong> All futures awaited inside the async</code> context implement the trait.</li>
</ol>
Any async fn</code> or async {}</code> block would automatically be able to implement
AtomicFuture</code> if those requirements are upheld, which I believe is something
which shouldn't be too hard to figure out from the generated state machine
3</a></sup>. Maybe there is a case to be made for syntax for this too; I'm not
sure. But that's something we can figure out later.</p>
^{3</sup>
Also: this is something we'll want to do regardless if we ever
get generator functions. It would be pretty bad if gen fn</code> couldn't
automatically implement marker traits on the type it returns.</p>
</div>}// ✅ -> impl Future + AtomicFuture
</span>async </span>fn </span>foo</span>() -> </span>u32 </span>{ </span>12 </span>}
</span>
</span>// ✅ -> impl Future + AtomicFuture
</span>async </span>fn </span>foo</span><F: AtomicFuture>(</span>fut</span>: F) -> </span>F::</span>{
</span>    fut.await
</span>}
</span>
</span>// ❌ -> impl Future
</span>async </span>fn </span>foo</span><F: Future>(</span>fut</span>: F) -> </span>F::</span>{
</span>    fut.await
</span>}
</span>
</span>// ❌ -> impl Future
</span>async </span>fn </span>foo</span><F: AtomicFuture>(</span>fut1</span>: F, </span>fut2</span>: F) {
</span>    fut1.await;
</span>    fut2.await;
</span>}
</span></code></pre>
The compiler will only be able to automatically figure out whether
AtomicFuture</code> can be implemented for futures returned by async fn</code> and
async {}</code>. For manually implemented futures the author of the future will need
to uphold those guarantees themselves. The bar to meet there is that the future
should only perform a single operation, and then return. read</code> is a good
example of a "stateless future", while the future join</code>
operation</a>
is a good example of a future which is not.</p>
As a basis I think this is pretty good, but there are still some cases we
haven't covered yet. For example tokio's Mutex::lock</code>
operation</a>:
only performs one operation, but when it's dropped and recreated it'll be the
last in the unlock queue again - which in pathological cases could lead to
certain unlocks never being scheduled. It's not marked "cancellation-safe" in
the tokio docs, but the reason for that is not covered by the rules we've
described so far.</p>
I think practically we'll want to play around with the rules a little. I don't
think we want or even can practically nail down what a "side-effect" is in Rust.
But if we intentionally don't make the presence of AtomicFuture</code> a safety
invariant, we can probably get away by adding broad rules such as:</p>

"Cancelling and recreating an AtomicFuture</code> should not cause logic errors"</p>
</blockquote>
What does that exactly mean? That's up for interpretation. But that might be
fine for our purposes. I think it's worth experimenting a little, so we can
settle on something that works for us. Ideally a definition which can be
automatically implemented by the compiler for async blocks and functions, but if
we can't that's probably also okay. The key takeaway here should be that if we
care enough about "cancellation-safety" to make it a guarantee of our public
APIs, we should be trying to bring it into the trait system instead.</p>
Pin Ergonomics</h2>
I think if we're to have an honest conversation about poll-based APIs, we should
acknowledge that working with Pin</code> is not a great experience for most people.
I'm a world-leading expert in async Rust, and four years post stabilization I
still regularly struggle to use it. I've seen folks on both T-lang and T-libs
struggle with it. I've seen very capable engineers at Microsoft struggle with
it. And if experienced folks struggle to use Pin, I don't think we can
reasonably expect those with less experience to use it without much problems either.</p>
I've covered some of the unique difficulties of Pin</code> on this blog before. I
believe pin is hard to use in part because pin inverts Rust's usual semantics
via a combination of auto-traits and wrapper structs. Another reason is because
it relies heavily on the concept of "pin-projection" which requires unsafe</code> and
is not directly represented in either the language or the stdlib. Which in turn
means that even the most common interactions with Rust's pinning system rely
third-party ecosystem crates such as
pin-project</a> and until
recently pin-utils</a>.</p>
We currently don't have a clear path to fix Pin's ergonomics. If we want Rust to
provide a way to make pin projections safe, we'll at least need to make a
breaking change to the Unpin</code> trait 4</a></sup>. As well as a whole lot of
design
work</a> to
integrate it into the language. And while this may be something we'll want to do
eventually, we really need to ask ourselves whether this is something we want to
do now</em>. Because taking on one project necessarily means not taking on another.</p>
^{4</sup>
Safe pin projections in the language necessarily require that
Unpin</code> is an unsafe</code> trait. It's currently not unsafe, and changing it would
be a breaking change.</p>
</div>
I honestly think async functions in traits, async drop, async iteration, and
async closures all are far more important things to work on right now, and they
should take priority in WG-async over directly fixing Pin's rough edges. The
most effective strategy to reduce the cost of Pin</code> we have in Rust right now is
to simply make pin less common</strong>. If Pin</code> is only needed to implement intrusive
collections, self-referential types, and manual future state machines then for
the time being we think of it as an experts-only system. And once we have more
bandwidth we can revisit it later to tidy it up.</p>}Evolution</h2>
A thing that worries me in particular about fn poll_next</code> is its inability to
co-evolve with Rust's needs. As we've seen we're not just considering adding an
"async version of iterator"</a>, we're also talking about "a pinned version
of iterator". But there are other flavors of iterator being discussed as well,
such as: "a fallible version of iterator"</a>, "a lending version of
iterator"</a>, "a const version of iterator", and so on. It's impossible to
know today which requirements Rust will have ten years from now. All we really
know is that the decisions we make then will be affected by the decisions we
make today 5</a></sup>.</p>
^{5</sup>
Ten years ago we were unsure whether it was even possible to publish
a new systems programming language. Five years ago we were unsure whether Rust
would be able to escape its browser niche and reach mainstream adoption. Flash
forward to today, and Rust is being used in the hearts of major operating
systems (Android, Linux, and Windows). As well as every packet on the internet
more likely than not being routed through some Rust code at some point
(Cloudflare, AWS, and Azure). I don't think we can reasonably anticipate all the
requirements Rust will face ten years from now. So I believe one of the most
important things for a programming language is to find ways to keep evolving the
language in the face of changing requirements.</p>
</div>}Say for a second we did add an fn poll_next</code>-based version of iterator. If we
wanted to say, add a lending version of iterator. Then we'd probably also want
to add a lending version of async iterator</a> too. That would be four
separate families of traits, and that's just for three variants. If we actually
wanted to add support for "pinned iterators" or "fallible iterators" we'd be
looking at nine or maybe even seventeen different families of traits. And nobody
reasonably wants that.</p>
This is the reason why I believe async fn next</code> is the better direction. If we
add it to the stdlib via an effect-generic mechanism, both "iterator" and "async
iterator" could be served using a single trait which is generic over the async
effect. But we could use the same mechanism for other traits such as Read</code> and
Write</code>, but also maybe some less common traits like Into</code> and Display</code>.</p>
//! A version of iterator which can be implemented as either sync or async.
</span>//! This uses placeholder syntax just to illustrate the idea.
</span>
</span>#[</span>maybe_async</span>]
</span>trait </span>Iterator {
</span>    </span>type </span>Item;
</span>    #[</span>maybe_async</span>]
</span>    </span>fn </span>next</span>(&</span>mut </span>self</span>) -> Option<</span>Self::</span>Item>;
</span>}
</span></code></pre>
fn poll_next</code> forces us to duplicate existing interfaces in order to expose
those same capabilities. While async fn next</code> enables us to extend existing
interfaces with new capabilities. It doesn't immediately solve all of the
limitations of iterator; effect generics won't provide an answer for how to add
support for "lending" or "pinned" iterator variants. But it provides an answer
to some other problems, like how to add support for async, const, and
fallibility. And in doing so encourages us to find similar solutions for the
problems which remain, even if we don't yet know what they'll look like.</p>
Conclusion</h2>
In this post I've presented a case for basing the async Iterator trait on async fn next</code> over fn poll_next</code>. To summarize the arguments:</p>

Performance</strong>: Iterators based on async fn next</code> and fn poll_next</code>
generate identical code, which means they have identical performance profiles.
Implementations which need to access the low-level future state machine of
async fn next</code> can do so using poll_fn</code></a>.</li>
Self-referential iterators</strong>: fn poll_next</code> is the async version of
PinnedIterator</code>, async fn next</code> is the async version of Iterator</code>. Adding a
trait for pinned async iteration could be useful, but realistically it should
mirror a synchronous "pinned iterator" design. And it could be written using
async fn next</code> taking a pinned self-type.</li>
Unpin bounds</strong>: The presence of extra Unpin</code> bounds are a way in which
async traits presently meaningfully deviate from their synchronous counterparts.
"async iterator" meaningfully deviates from its synchronous
counterparts. It makes it seem like deeper changes are needed to get to the
same async functionality. By not requiring self: Pin<&mut Self></code>, no
additional where Self: Unpin</code> bounds are required as seen on methods such as
StreamExt::next</code></a>.</li>
Implementation vs Usage</strong>: async fn next</code> has one method to both
implement and use. fn poll_next</code> requires two methods: one which must be
implemented, and another which must be called. This is inherently more
difficult to use, and as a mechanism is unique to poll functions other than
Future</code>.</li>
Object Safety</strong>: async fn next</code> and fn poll_next</code> need to use subtly
different mechanisms to create dynamically dispatched objects. Adding support
for dynamic dispatching is still in-progress for AFITs, but that seems like
it's mostly a matter of time. Neither approach is likely going to be much
harder to use than the other, but the subtle differences may be difficult to
internalize for users. Diagnostics seem like they'll play an important role.</li>
"Cancellation Safety"</strong>: "cancellation safety" is currently a
documentation-only property. If async iterator is based on fn poll_next</code> users
can blindly assume every implementation provides a cancellation-safe next</code>
method. If async iterator is based on async fn next</code> users will have to check
the implementation's docs to learn whether the next</code> method is
"cancellation-safe". However, rather than keeping "cancellation-safety" as a
documentation-only property, we should probably instead be working to bring
into the type system instead.</li>
Pin Ergonomics</strong>: The Pin</code> family of APIs in Rust is notorious for being
difficult to use. One of the most effective ways we have to reduce the
difficulty of Pin is by limiting user's exposure to it. By basing Rust's core
async traits on async functions we can reduce the number of pin-based APIs,
making async Rust more accessible to more people.</li>
Evolution</strong>: async fn next</code> can be implemented as an extension</em> to the
existing iterator trait via effect generics. fn poll_next</code> would most need to
be implemented as a standalone trait, resulting in a manual duplication</em> of
the APIs. Support for async is not the last feature we'll want to add to
iterators: there are currently ecosystem demands to support self-referential
iteration, lending iteration, and fallible iteration. It's not practical to
add individual traits for all of these and their combinations. Nor is it
reasonable we can anticipate all needs which may arise in the future. By
extending rather than duplicating we</li>
</ul>
I believe basing the async Iterator trait on async fn next</code> to be superior
across all axes, and I hope this post sufficiently makes that case. In a future
post I'd like to round out this series by covering the desugaring of an async
iteration notation. Together with an RFC for effect-generic trait definitions,
this will be one of the last steps necessary step before we can re-RFC RFC
2996: Async Iterator</a>
to cover the full scope of async iteration.</p>
Thanks to Eric Holk for reviewing an earlier draft of this post.</em></p>

nesting allocators
Wed, 09 Aug 2023 00:00:00 +0000
I regularly point people to Tmandry's 2021 post "Contexts and Capabilities in
Rust"</em></a>.
While nobody is actively working on the design at the moment, it's more because
of prioritization than anything else. It seems like it could make certain usage
patterns a lot nicer in Rust, and one of those is probably working with custom
allocators.  I've been working a lot with slab
allocators</a> recently, and I'd
love it if they were easier to work with in Rust. So I wanted to take a moment
to talk about allocators, capabilities as a language feature, and why I believe
that would work well.</p>
Kinds of allocation</h2>
Memory, at least semantically, is best thought of as a contiguous sequence of
bytes. Allocators manage these bytes and do the bookkeeping for who is using
what. Allocators can take any type of object ("heterogenous types") and place
them in memory. And once we're done with an object, the allocator will reclaim
the memory. Because not all objects are the same size this can leave gaps in
memory, so occasionally the allocator spends some time reorganizing its
internals (think: defragmentation). In practical terms, every program wants to
have a single allocator to manage the entire</em> program. This root allocator is
what we refer to as the global allocator</em>.</p>
But where global allocators are great</em>, they are not perfect</em>. Typically when
we write programs we'll see usage patterns in our main loop. For example: if
we're writing an HTTP server we will typically allocate data for the request
headers and body. And once that request is handled, we de-allocate it and
allocate a new request. We can use our knowledge of our usage patterns to handle
allocations in a more targeted way. The way we do this is by obtaining a
relatively large-ish chunk of memory from our system allocator</em>, and then using
that as the heap space for our custom allocator. These custom allocators are
also known as "region-based allocators"</em> or "arena allocators"</em>. Their key
benefit is that their memory management strategies are very simple, which can
make allocations so cheap they might as well be free.</p>
There are many different types of arena allocators, but these are the ones I use
the most:</p>

slab allocators:</strong> For example the slab</a>
crate in Rust. This is ideal for when you perform many homogenous, short-lived
allocations over a longer period.</li>
bump allocators:</strong> For example the bumpalo</a> crate
in Rust. This is ideal for phase-oriented allocations</em>, where you allocate a
number of heterogenous objects, say, on every iteration of a loop and then
de-allocate them at the end of it.</li>
</ul>
While not every crate lends itself equally well to it, it's even possible to
combine arena allocators to further customize allocation behavior.</p>
Nesting Allocators</h2>
The bumpalo</a> crate comes with an optional collections</code>
feature</a> which
can be enabled to grant access to String</code> and Vec</code> types which are identical
to the stdlib's versions other than the fact that they allocate on the bump
allocator. Using it looks like this:</p>
use </span>bumpalo::{Bump, collections::Vec};
</span>
</span>let</span> b = Bump::new();                 </span>// Create a new bump allocator
</span>let</span> v: Vec<</span>i32</span>> = Vec::new_in(&b);   </span>// Allocate on the bump allocator
</span></code></pre>
This is an example of nesting allocators</em>. The system allocator hosts the
bump allocator, and the bump allocator in turn hosts the vector. The main
downside of this is that we're not using std::vec::Vec</code>, but have to use
bumpalo's custom vec implementation. That's not ideal, not just because we now
have a second Vec</code> type, but because we also don't have other collection types
such as HashMap</code>, HashSet</code>, and every other type out there on crates.io.</p>
Passing Allocators</h2>
The plan is to solve this limitation via the nightly allocator_api</code> feature in
the stdlib. This extends the stdlib with methods such as
Vec::new_in</code></a>
which can take a custom
Allocator</code></a> argument
to host the allocations in. I don't have any particular opinions about the exact
Allocator</code> trait, but I do have opinions about the way this bubbles up to
collection types.</p>
Roughly speaking, the current design adds a new allocator generic param to
collection types which defaults to the global allocator (Vec<T, A = GlobalAllocator></code>). In order to uniformly make use of this, every constructor
method for every collection type needs to provide new _in</code> variants of their
existing collections. At first that seems reasonable; but it quickly runs into
issues.</p>
The first issue is usage with macros, which we can probably overcome. Macros
like vec!</code> perform allocations, and in order to work with custom allocators
they should probably take an optional third param for the allocator. The second
issue is a lot harder: traits can't take extra arguments. For example, if you
have an &str</code>, and you want to convert it to String</code> but use a bump allocator
you can't keep using the existing into</code> method. And because the into</code> trait is
part of core</code>, we can't just add a new into_in</code> method to the Into</code> trait
either. The only real option is probably a new trait in alloc</code>:</p>
let</span> s = "</span>hello</span>".</span>to_owned</span>();         </span>// 1. Uses the global allocator
</span>let</span> bump = Bump::new();
</span>let</span> s = "</span>hello</span>".</span>to_owned_in</span>(&bump); </span>// 2. Uses a bump allocator
</span>
</span>let</span> s: String = "</span>hello</span>".</span>into</span>();     </span>// 1. Always uses the global allocator
</span>                                    </span>// 2. ❌ Unclear how to create a variant
</span>                                    </span>//    of this with a custom allocator
</span></code></pre>
While the number of collection types and constructor methods are limited, things
can quickly spiral out of hand when we factor in composition</em>. In order for
things to work uniformly across the ecosystem, every type which holds a Vec</code>
internally would need a second _in</code> variant for all of their constructors and
conversions. That kind of duplication is uncompelling, meaning it's likely we're
only going to see some</em> types adopt support for custom allocation while the
majority doesn't. Not unlike how #[no_std]</code> is not uniformly supported
throughout the crates ecosystem today.</p>
Scoped Allocators</h2>
What we're really struggling with is that the global allocator acts, in a
capability-sense, as an ambient
authority</a>. Which is to say:
every function in Rust has access to the global allocator without functions
needing to take allocators explicitly as arguments. The allocator is just always
present. Except when we're in #[no_std]</code> environments, where there is no
allocator. Or when we're trying to change which allocator we're trying to take.</p>
The _in</code> family of methods is an attempt to remedy the issue of having an ambient
allocator. But that has some drawbacks: most notably the bifurcation of APIs.
But also: it seems like that would be quite annoying to use. Manually having to
pass some form of alloc</code> argument down to every function call doesn't exactly
seem ideal.</p>
Instead Tmandry's capabilities design</a> seems like a much more pleasant direction.
It acknowledges the existence of ambient capabilities, and instead just asks
that we express them at the signature level. Because they effectively function
as implicits, they just carry through to all argument in scope which need them:</p>
let</span> s = "</span>hello</span>".</span>to_owned</span>();             </span>// 1. Uses the global allocator
</span>with alloc = Bump::new() {              
</span>    </span>let</span> s = "</span>hello</span>".</span>to_owned_in</span>(&bump); </span>// 2. Uses a bump allocator
</span>}
</span>
</span>let</span> s: String = "</span>hello</span>".</span>into</span>();  </span>// 1. Always uses the global allocator
</span>with alloc = Bump::new() {
</span>    </span>let</span> s = "</span>hello</span>".into;        </span>// 2. ✅ Uses a bump allocator       
</span>}
</span></code></pre>
Compositions with functions and types would no longer require manually passing
allocators into arguments, but merely acknowledging the existence of an
allocator in the first place. That would mean we don't have to provide new _in</code>
methods which take allocators, but instead we could keep using the existing
new</code> methods with some minor modifications.</p>
// Vec's design on nightly today, providing both `new` and `new_in` methods
</span>impl </span>Vec<T, A = Global> {
</span>    </span>fn </span>new</span>() -> </span>Self </span>{ ...  }
</span>}
</span>impl </span>Vec<T, A> {
</span>    </span>fn </span>new_in</span>(</span>alloc</span>: A) -> </span>Self </span>{ ...  }
</span>}
</span>
</span>// A design of vec using capabilities, using just a single `new` method.
</span>impl </span>Vec<T>
</span>with alloc = Global,
</span>{
</span>    </span>fn </span>new</span>() -> </span>Self </span>{}
</span>}
</span></code></pre>
Obviously questions remain about how to exactly model capabilities. Many types
in the stdlib liberally make use of ambient capabilities (e.g. allocation,
filesystem, networking, etc.) which we'd need to continue allowing to work as it
does today. But there is a question about how we can instead weave through
user-provided capabilities through the entire stdlib and crates.io ecosystem.
The answer requires having a way to continue relying on ambient capabilities,
but opt-out if you want to. Which is not entirely unlike const</code>.</p>
The final benefit to this is that this would solve one of the current blockers
we have for string-literals. The obvious design for that would be to allow the
s</code> prefix on literals to create allocated strings. But because that would act
as a language built-in, there is an open question on how to make that work with
custom allocators. Using capabilities would provide a compelling solution for that:</p>
// Rust today
</span>let</span> s = String::from("</span>Chashu</span>");
</span>
</span>// With an `s` prefix, allocating on the global allocator
</span>let</span> s = s"</span>Chashu</span>";
</span>
</span>// With an `s` prefix, allocating on a local allocator
</span>with alloc = Bump::new() {
</span>    </span>let</span> s = s"</span>Chashu</span>";
</span>}
</span></code></pre>
Conclusion</h2>
In this post we took a look at general allocation strategies, how allocations
can be nested today, which problems that approach has, and how we could simplify
that via the novel contexts/capabilities language-feature. The design of
contexts/capabilities is incomplete, so we intentionally didn't go too deep into
that. Instead we only touched on the top-level semantics, showing specifically
how it could be leveraged to overcome some of the problems we have with
allocation today.</p>
Fundamentally capabilities don't provide anything new</em>; they're just a fancy
way of passing arguments to functions. But I also believe that is their
strength. It is just passing arguments to functions</em>. But with significantly
less code to write, no method duplication, the ability to play along with trait
impls, and a lot more clarity of what your function needs to function</a>.</p>
I don't just think that capabilities could simplify the problem space for
allocators in Rust. I actually believe that it could be the canonical way in
which we solve the current core</code>/alloc</code>/std</code> split in Rust. The core</code>
library really is just std</code> minus everything which relies on ambient
capabilities. If we could opt-out of relying on ambient capabilities in std</code>,
the reason to have separate core</code>, alloc</code>, and std</code> libraries should go away.</p>
The other thought I want to share is that I believe capabilities such as
allocators are inherently tree-structured</em>. I've recently written about
tree-structured
concurrency</a>
explaining how concurrency is best thought of as a tree. But from the
operational semantics side of the language we now
tree-borrows</a> as well.
I feel like at a high-level thinking about program execution as a tree is
accurate, and we should be thinking of (nesting) capabilities such as allocators
as trees as well.</p>

Totality
Thu, 20 Jul 2023 00:00:00 +0000
Recently Eric</a> and I have been chatting with Daan
Leijen</a>, of
Koka</a> fame, discussing among
other things the distribution of effects. Daan estimated that for a typical
program written in Koka the distribution of effects would roughly be something
like:</p>

~70% of a program could be "total" (as in: has no effects)</li>
~15% of a program would have to deal with exceptions or divergence (may error,
may panic, or may loop indefinitely)</li>
~5% of a program ends up dealing directly with IO resources</li>
</ul>
That leaves another ~10% for wiggle room and other effects. And I think the
broad implications of this are fascinating</em>. At least for Koka, according to
Daan the vast majority of code could be written as entirely un-effectful. No
panics, no infinite loops, no IO - just functions operating on input and
producing output.</p>
total functions</h2>
Perhaps you may have heard people refer to Haskell as a "pure" language.
Functions in Haskell may by default diverge (loop infinitely), and they may
throw exceptions. Koka on the contrary is what you could call a "total
language". By default functions are considered "total</em>" 1</a></sup>, and are
only allowed to operate on their inputs, and produce outputs. In Koka if you
want to write a "pure" function, you write a function which makes use of the
exn</code> effect ("may throw exceptions"</em>) and div</code> effect ("may diverge"</em>):</p>
^{1</sup>
I believe terminology here is plucked from math/category theory, which
I know little about - so I hope I'm doing this description justice.</p>
</div>}// Has no side-effects ("total")
</span>fun </span>square_total</span>(</span>x</span>: </span>int</span>): </span>total int </span>{
</span>    </span>x </span>* </span>x
</span>}
</span>
</span>// Doesn't terminate + throws exceptions ("pure")
</span>// "pure" is an alias for the `<div,exn>` effects
</span>fun </span>square_pure</span>(</span>x</span>: </span>int</span>): </span>pure int </span>{
</span>    </span>throw</span>("</span>oops</span>");
</span>    </span>x </span>* </span>square_pure</span>(</span>x</span>)
</span>}
</span></code></pre>
In Rust however, we can't write total</em> functions. While we are able to write
functions which meet all the conditions to be total</em> - we can't actually
express in the type system that they're total. Meaning that by reading a
function signature you have no idea whether a function will terminate, may
panic, or perhaps open files - it doesn't say.</p>
We get pretty close to the "pure" set of effects through the const fn</code>
notation. It can't heap-allocate, it can't access IO, and can't access globals
("statics"). Right now it's also deterministic</em> - in Koka if code wants to be
nondeterministic it needs to be marked as
ndet</code></a>.  But in
order to declare "total" functions, we'd need to be able to strip the set
effects on functions further in Rust beyond what const fn</code> does:</p>

The ability to declare functions which will not panic</li>
The ability to declare function which "will not diverge" (guarantee termination)</li>
</ol>
So let's take a look at what that means, and what it would take to implement in Rust.</p>
Functions which cannot panic</h2>
Rust distinguishes between "panic" for programmer-errors and "result" for
runtime-errors. Both can be mapped into each other using a combination of:</p>
</th> result</th> panic</th></tr></thead>

Create</td> return Err(..)</code></td> panic_any</code></a> / unwrap</code></a></td></tr>
Consume</td> match {}</code></td> catch_unwind</code></a></td></tr>
</tbody></table>
However, despite it being possible to convert between panics and errors - panics
themselves do not exist anywhere in the Rust type system. You cannot construct a
panic type and pass it around, you can only trigger a panic in-place. Similarly
functions don't return a Panic<T></code> or impl Panic</code> when they can panic. Every
function can panic, and the ability to panic lives outside of the type system. 2</a></sup></p>
^{2</sup>
I don't think it's necessarily bad that panics live outside of the type
system though. I'm definitely not convinced that we should bring it into the
type system? Maybe it's closer to const</code> in the way that it just annotates the
function, but it doesn't live in</em> the type system? idk.</p>
</div>}"But Yosh?", I hear you asking, "Do people even want</em> to write functions which
can't panic?" And well dear reader, people certainly want to be able to do that.
There's a thread full of people
here</a>
discussing wanting the ability to opt-out of panics. For reasons such as better
FFI (unwinding through FFI boundaries is generally unsound in Rust), support for
embedded systems, and more. Formal verification tools such as
Prusti</a> and
Kani</a> advertise their ability to prove
the absence of panics as their key features. And I know some folks working on
Windows here at Microsoft who would love it if we could prove that code which
should not panic in fact will never panic.</p>
A basic form of "will not panic" notations can be implemented today via third
party crates such as Dtolnay's no-panic</a>,
and Japaric's panic-never</a>. But while
it works okay, code like that is not composable and will eventually run into
the effect sandwich problem</a>.
Instead the better approach is probably to bring it into the type system, and
reason about it as an effect.</p>
While a "do not panic" notation is sufficient to prove that a function won't
panic, inside the function body we're then obliged to actually write code which
won't panic. And even with notations that's hard because of a number of reasons:</p>

In Rust any operation may currently panic - opting into "will not panic"
semantics would be a pretty big shift.</li>
Proving panic-freedom in a way that is also ergonomic requires introducing
verification tools into the language. Pattern
types</a>
backed by something like Flux</a>
could be one approach. But this is an incredibly rich topic with lots of
challenges - many of which we don't know if we can overcome.</li>
</ul>
In my post on pattern
types</a> I go into
this with some more detail on how we could deal with the first issue. Roughly
said: if we were able to encode more details about the types in the types
themselves, we could use that to prove to the compiler certain operations are
fine:</p>
// ❌ A basic doubling function in today's Rust
</span>// combined with a "no_panic" notation
</span>//
</span>// Compiler error: this could panic because: u32::MAX + u32::MAX >= u32::MAX
</span>#[</span>no_panic</span>] </span>fn </span>double</span>(</span>num</span>: </span>u32</span>) -> </span>u32 </span>{
</span>    num + num
</span>}
</span>
</span>// ✅ A basic doubling function using a hypothetical "pattern type"
</span>// notation combined with a "no_panic" notation
</span>const</span> N: </span>u32 </span>= </span>u32</span>::</span>MAX </span>/ </span>2</span>;
</span>#[</span>no_panic</span>] </span>fn </span>double</span>(</span>num</span>: </span>u32</span>[0..N]) -> </span>u32</span>[</span>0</span>..</span>u32</span>::</span>MAX</span>] {
</span>    num + num
</span>}
</span></code></pre>
functions which guarantee termination</h2>
In Rust by default functions do not guarantee termination. And that's not a bad
property: we want</em> to be able to write functions which don't guarantee
termination, such as HTTP servers which keep listening indefinitely for
requests, or other similar "main loops". Useful stuff.</p>
But if every part of your program can loop indefinitely, it acts a little bit
like if any part of your program has access to raw pointers or IO: it can be
hard to track where exactly things are going wrong, because the potential site
of it is undetermined.</p>
Just like with the hypothetical "no_panic" notation, we could have some form of
a "nobody_loops" 3</a></sup> notation. This could signal to the compiler
that a function will guarantee termination, and that actions which don't
terminate are disallowed. This one is undecidedly hard</del> impossible to prove for
the general case, but if we wanted to do something practical we could probably
create a new unsafe marker trait like say BoundedIterator</code> which would be
automatically implemented for things like looping over ranges, looping over
items in collections, etc.</p>
^3</sup>https://github.com/rust-lang/rust/pull/19964</a></p>
</div>
// ✅ This would compile fine since the loop in the body has an upper bound
</span>#[</span>nobody_loops</span>] </span>fn </span>main</span>() {
</span>    </span>for</span> n in </span>0</span>..</span>100 </span>{
</span>        println!("</span>{n}</span>");
</span>    }
</span>}
</span></code></pre>
Similar to Rust's borrow-checking rules, it's unlikely any implementation for
"will terminate" will be enough to ever allow the full set of hypothetically
compliant programs.  That requires proving termination in the general case,
which I believe may be impossible. Instead the focus should be on a practical</em>
subset of programs which can provably terminate. This is another potential
reason why generators as a first-class language feature would be useful. Because
the compiler generates the iterator implementation, it would enable creating
terminating iterators without manual unsafe trait impls:</p>
// Made-up syntax. This would return:
</span>// -> impl Iterator<Item = Cat> + BoundedIterator
</span>#[</span>nobody_loops</span>] gen </span>fn </span>cats</span>() -> Cat { .. }
</span></code></pre>
Const Functions</h2>
In Rust the const</code> effect isn't so much an effect onto itself, as it is the
subtraction of effects from the "base" set of effects. As we mentioned: const
functions must (currently) be deterministic, don't get access to IO, don't get
access to statics, and can't allocate on the heap. And in return they gain the
ability to be evaluated during compilation.</p>
But that's if we view it from the perspective of Rust's base set of effects. If
we look at const from a "total" perspective, then "const" represents roughly
represents the "may panic" and "may diverge" effects. And while we don't have
any annotations for basic functions in Rust, they effectively act as a superset
of "const", and also gain access to globals, the host environment (io), etc.</p>
So if we take a position that const functions somehow subtract effects from
"base Rust", then it is indeed the case that "const" can probably not be
considered an effect. But if we take a look at the effects from a perspective of
totality, then "const" and "base" contexts simply represent different sets of
effects:</p>

total</strong>: no effects</li>
const</strong>: panic + diverge</li>
base</strong> 4</a></sup>: panic + diverge + allocate + host environment + globals</li>
</ul>
^{4</sup>
By "base Rust" I mean the capabilities a function has
when you write fn foo</code> with no other annotations, compiled for a std-capable
target. It doesn't have any specific effect annotations like const</code> or async</code>,
hence the "base" qualifier.</p>
</div>
Reasoning about effects using a starting point of totality is also far simpler.
It might not necessarily map to the surface-level syntax Rust provides, but it
certainly makes it easier to reason about what happens when say, you have an
unsafe const fn</code> compared to a async gen fn</code> or something. Both have added
effects on top of "total", but while they slightly overlap they also differ
somewhat. This makes it easier to reason about than when only talking about
effect in terms of subsets or supersets, rooted in "base" Rust.</p>}Evolving Totality</h2>
So far we've mainly focused at the "may panic" and "does not guarantee
termination" effects. We've made the case that if only we had both those
effects, we'd achieve totality. But that's not quite true. What "total" means
for a language will differ depending on the language. In a way you think of
"total" as the absence of all optional effects, but it includes all non-optional
assumptions.</p>
Over time we can discover we want more control over which assumptions are baked
in and are optional. We've seen this with "const rust" already, which provides
fewer effects than "base rust" does. And both "may panic" and "may diverge" are
built-in assumptions which exist in Rust today, but could potentially be
surfaced. But those aren't the only ones, there are potentially more such as:</p>
⚠️ Reader alert: I'm not proposing we add any of these effects. These are just
intended as illustrative examples for a point I'm working up towards the end of this
section. ⚠️</em></p>

Access to host environment</strong>: every function in Rust assumes it has access to a host
environment, and it can just call things like File::open</code> to access resources
on the host. Embedded systems can't assume a host. The difference between the
two is currently not encoded in the type system, and a notion of "total" may
want to do away with the assumption a host is present.</li>
Drop is not guaranteed to be called</strong>: today any function in Rust can freely
take any type and pass it to mem::forget</code> or similar to leak them without
calling their destructors. In a way you can think of every function having
access to a leak
effect</a>, and a notion of
"total" may want to exclude that.</li>
Type conversions can be reversed</strong>: say a function takes an impl Read</code>
argument, using the Any</code>
trait</a> it's possible to obtain
the underlying type, as long as the type name is accessible. This means there
is no guarantee that a function will only operate on the impl Read</code>
interface, because it may in fact cast it back to the original type and
operate on that instead. Being able to obtain the underlying type could be
considered an effect which "total" may not want to have access to.</li>
Type sizes can be accessed</strong>: every type has a size and a layout. Functions
can learn information about a type by calling the
size_of</a> function on a
type. A notation of totality could restrict obtaining any size or layout
information about a type.</li>
Memory locations can be directly accessed</strong>: we assume types are backed by memory
addresses in the address space. This means we can always find the memory
address of a type. But what if we were to run, say, Rust on the JVM. The JVM
doesn't expose an address space; it exposes objects and references. Which
means that "type locations can be accessed" could be considered an effect,
which could be absent from totality by default.</li>
</ul>
And there are probably lots more potential "effects" which fall outside of the
notion of "totality" presented in this post so far. That's because in a way
writing effects brings the assumptions we make about functions directly into the
type system. And because our programs run on real physical hardware with memory
models and operating systems, it's possible to map just about any detail of the
underlying hardware into an effect. Even to the point of silliness, such as
an effect which enables you to allocate stack space.</p>
Choosing which effects to add means choosing which assumptions to surface. That
is a real art, and requires thinking through the purpose and use cases of the
effects. Ideally effects could even come in themes: for example "type
conversions can be reversed" and "type sizes can be accessed" share a theme of
"type privacy" - maybe they could be folded into each other. We've done
something similar with "const" too, where we've combined "panic" and "diverge"
into a single effect. What I'm trying to say is: what we mean by "total" depends
on our current idea of what the language is and should do. And it's even
something which can change over time as our model and needs of the language
evolve.</p>
Conclusion</h2>
Total functions go a step beyond "pure" functions because they also disallow
code from panicking and require functions provably terminate. Effect-wise it
represents the least a function could possibly do, so when talking about effects
we can think of every other effect as "adding" effects on top of the "total"
effect. This will also be helpful in case we ever manage to constify</em> the
majority of the stdlib, and want to switch the default set of implied effects
across an edition bound. Reasoning about effects rooted in totality provides us
with a model to reason about what would change.</p>
As we're closing out here, something I did want to briefly touch on Rice's
theorem</a>. It states that: "All
non-trivial semantic properties of programs are undecidable".</em> In effect what
we're doing with effects is attempting to go against that. The unsafe</code> keyword
in Rust exists because the compiler can't prove memory safety for all</em>
functions, just most of them but we need a way out for when we can't. Similarly
we won't be able to prove panic-freedom for all functions, or prove all
functions can terminate. But we can for a lot of functions; and statically
proving interesting properties of our programs is kind of what type systems are
about.</p>
I find it fascinating that ~70% of code in a typical Koka program can be marked
as total. For Rust there is a thriving research movement to build verifiers for
the language, and one of the properties they often start by proving is
panic-freedom for functions. I can't help but wonder how we could not just
expose these kinds of guarantees through extensions to Rust, but actually by</em>
Rust itself. And I think in order to do that, we may want to start with by
adding ways to reason about semantic properties such as "may panic" or "may
diverge" as effects native to Rust.</p>
Thanks to Eric Holk</a> and Daan
Leijen</a> for reviewing
this post prior to publishing, and chatting about effects over the past few
months.</em></p>

bridging fuzzing and property testing
Mon, 10 Jul 2023 00:00:00 +0000
It's been over three years since Fitzgen published: "Announcing Better Support
for Fuzzing with Structured Inputs in
Rust"</em></a>,
and a little over two years since
arbitrary</a> 1.0 was released. A few years agoI
wrote a property-testing crate based on arbitrary</code>, but never ended
up writing about it. So I wanted to take a moment to change that. We'll start by
taking a look at automated testing, then cover the different approaches to it,
explain how to work with structured inputs, and finally introduce heckcheck</a> -
a small property-testing library.</p>
Update 2023-07-18:</strong> I recently learned about a different crate using a
similar approach to mine:
proptest-arbitrary-interop</a>.
The proptest crate is more mature than my heckcheck</code> project, so if you're
considering adopting the techniques described in this post - please check out
the proptest interop crate too.</p>
fuzzing vs property testing</h2>
Both fuzzing</a> and property
testing</a> are ways of
automatically testing code. Where unit tests typically test some expected set of
behavior, automated test have the ability to be more rigorous and exhaustive -
making them far more likely to weed out bugs - especially those you didn't even
conceive of. Fuzzing and property testing have a lot in common with each other.
Where they tend to differ is how tests are driven.</p>
With fuzzing you typically use some external agent to test your program. Fuzzers
usually have the ability to instrument the code under test, and make use of
tools such as
sanitizers</a> to
check whether invariants are violated. Fuzzers will also keep track of which
input leads to which branches being hit in code, tailor their input to cover as
many branches as possible. This process can take time, and is why it often pays
off to run fuzzers for extended periods of time or even
continuously</a>. In Rust support for fuzzing
is provided via the
cargo-fuzz</code></a> extension.</p>
Property testing typically works the other way around: property testing is
typically done by including a library in your test code which allows you to
drive the tests yourself. Rather than looking for crashes or sanitizer failures
the emphasis is more on checking your implementation using a strategy - more on
those in a later section. While fuzzing tends to be more thorough, property
testing tends to be faster to execute. Depending on what you're testing it's not
uncommon to integrate property tests in your CI test suite - and run them every
time you make a change. If you're writing unsafe Rust you can even combine
property tests with miri</a> to validate with more confidence whether the runtime
behavior of your program matches Rust's operational semantics.</p>
The final piece of both property testing and fuzzing is that they perform
something called test case shrinking</em> when they're done. When they encounter a
failing test, what they'll do is try and simplify the input to the bare minimum
to isolate the issue as much as possible. Once simplified it tends to be much
easier to understand, and in turn can become easier to convert into unit
tests</a>.</p>
There's a lot more to say about automated testing - including how to use it to
mess with the ordering of concurrency primitives</a>, or to intentionally
mess with the network</a>. But roughly speaking that should all just be a
matter of wiring up drivers up to the same source of entropy coming from the
automated testing drivers.</p>
Structured Inputs</h2>
In Fitzgen's
post</a>
he shows how it's possible to combine fuzing with structured inputs using
arbitrary</code> crate. The way a fuzzer typically works is that it generates random
data which is passed to a program over some channel. Sometimes this data is
generated from some corpus of data to try and weave in phrases we know the
program might be looking for - such as HTTP/1.1</code> or GET</code> for an HTTP parser.
But with arbitrary</code> this randomness can be more clearly channeled: it can take
the arbitrary stream of data, and use it to create structured types.</p>
Part of the reason why Fitzgen did this work was to fuzz WASM programs. In
"Writing a Test Case Generator for a Programming
Language"</em></a>
he describes how using cargo fuzz</code> and arbitrary</code> he's able to generate
endless amounts of new, valid WASM programs to pass to be used to test all
sorts of WASM-related tools</a>. This allows
far more code to be exercised than solely using a corpus-based approach ever
could.</p>
To show an example of what it looks like to use cargo-fuzz, we can re-use the
RGB parser/encoder example from Fitzgen's blog - but with the actual methods
implemented. We start by defining a struct which takes red, green, and blue
values:</p>
use </span>arbitrary::Arbitrary;
</span>
</span>/// A color value encoded as Red-Green-Blue
</span>#[</span>derive</span>(Arbitrary, Debug, PartialEq, Eq)]
</span>pub struct </span>Rgb {
</span>    </span>r</span>: </span>u8</span>,
</span>    </span>g</span>: </span>u8</span>,
</span>    </span>b</span>: </span>u8</span>,
</span>}
</span>
</span>impl </span>Rgb {
</span>    </span>/// Convert from RGB to Hexadecimal.
</span>    </span>pub fn </span>to_hex</span>(&</span>self</span>) -> String {
</span>        format!("</span>#</span>{:02X}{:02X}{:02X}</span>", </span>self</span>.r, </span>self</span>.g, </span>self</span>.b)
</span>    }
</span>
</span>    </span>/// Convert from Hexadecimal to RGB.
</span>    </span>pub fn </span>from_hex</span>(</span>s</span>: String) -> </span>Self </span>{
</span>        </span>let</span> s = s.</span>strip_prefix</span>('</span>#</span>').</span>unwrap</span>();
</span>        Rgb {
</span>            r: </span>u8</span>::from_str_radix(&s[</span>0</span>..</span>2</span>], </span>16</span>).</span>unwrap</span>(),
</span>            g: </span>u8</span>::from_str_radix(&s[</span>2</span>..</span>4</span>], </span>16</span>).</span>unwrap</span>(),
</span>            b: </span>u8</span>::from_str_radix(&s[</span>4</span>..</span>6</span>], </span>16</span>).</span>unwrap</span>(),
</span>        }
</span>    }
</span>}
</span></code></pre>
We can expose this to cargo-fuzz</code> to perform a roundtrip-test by creating a new
file containing the following:</p>
// The fuzz target takes a well-formed `Rgb` as input!
</span>libfuzzer_sys::fuzz_target!(|</span>rgb</span>: Rgb| {
</span>    </span>let</span> hsl = rgb.</span>to_hsl</span>();
</span>    </span>let</span> rgb2 = hsl.</span>to_rgb</span>();
</span>
</span>    </span>// RGB -> HSL -> RGB is the identity function. This
</span>    </span>// property should hold true for all RGB colors!
</span>    assert_eq!(rgb, rgb2);
</span>});
</span></code></pre>
We can then execute it using cargo fuzz</code> by running:</p>
$</span> cargo fuzz run my_test
</span></code></pre>
Heckcheck: Property Testing using Arbitrary</h2>
As we've covered the arbitrary</code> crate works great when fuzzing. And because
fuzzing and property testing are very similar, you'd assume that it should be
possible to use arbitrary</code> for property-based tests too right? I couldn't find
a crate which did this, so I ended up writing one! Enter
heckcheck</a>, a really small property-testing library. If we take our earlier RGB
example and adapt it to use heckcheck, it will look something like this:</p>
/// Validate values can be converted from RGB to Hex and back.
</span>#[</span>test</span>]
</span>fn </span>rgb_roundtrip</span>() {
</span>    heckcheck::check(|</span>rgb</span>: Rgb| {
</span>        </span>let</span> hex = rgb.</span>to_hex</span>();
</span>        </span>let</span> res = Rgb::from_hex(hex);
</span>        assert_eq!(rgb, res);
</span>        Ok(())
</span>    });
</span>}
</span></code></pre>
And that's all we had to do. To test the code we just run cargo test</code> as
usual:</p>
$</span> cargo test
</span></code></pre>
The same code we would've written to interact with a fuzzer can now be
used for property testing too. As I've said earlier: both fuzzing and property
testing have more in common than not. The difference here compared to using a
fuzzer is that this can be run as part of CI without a problem, executes really
quickly, and is entirely portable. Fuzzers need to be installed on the system
first, typically execute more slowly, and some platforms such as Windows have
limited support for fuzzers.</p>
I wrote heckcheck</code> mostly to see whether it would be possible to write - and
the implementation is maybe ~400 lines or so in total. I bet it could be
improved, since in particular our shrinking code is less than ideal, while good
approaches are
known</a>.</p>
Compared to other popular property testing libraries heckcheck</code> is more similar
to quickcheck</code></a> than it is to proptest</code></a>. It has a simple interface, can
generate complex values quickly, and uses external shrinking strategies. The
proptest
docs</a>
contain a comparison between proptest and quickcheck, and that same comparison
could probably apply between proptest and heckcheck. The only place where the
comparison may differ is that the arbitrary</code> crate has support for customizing
generated values. Say we wanted to restrict the values generated in our RGB
implementation to only valid RGB values, we could change the implementation to
the following:</p>
use </span>arbitrary::{Arbitrary, Unstructured};
</span>
</span>/// A color value encoded as Red-Green-Blue
</span>#[</span>derive</span>(Arbitrary, Debug, PartialEq, Eq)]
</span>pub struct </span>Rgb {
</span>    #[</span>arbitrary</span>(with = color_range)]
</span>    </span>r</span>: </span>u8</span>,
</span>    #[</span>arbitrary</span>(with = color_range)]
</span>    </span>g</span>: </span>u8</span>,
</span>    #[</span>arbitrary</span>(with = color_range)]
</span>    </span>b</span>: </span>u8</span>,
</span>}
</span>
</span>/// Generate a valid RGB color value
</span>fn </span>color_range</span>(</span>u</span>: &</span>mut</span> Unstructured) -> arbitrary::Result<</span>u8</span>> {
</span>    u.</span>int_in_range</span>(</span>0</span>..=</span>255</span>)
</span>}
</span></code></pre>
If I could change one thing about the arbitrary</code> crate it's that I'd like to
see it support pattern-type notation inside the attributes. Patterns allow
primitive values to be restricted to a subset of their possible inputs, which
should be a fairly common use. With that we could imagine the RGB type could
instead be written like this:</p>
//! ⚠️ This is purely speculative and not currently supported ⚠️
</span>
</span>use </span>arbitrary::{Arbitrary, Unstructured};
</span>
</span>/// A color value encoded as Red-Green-Blue
</span>#[</span>derive</span>(Arbitrary, Debug, PartialEq, Eq)]
</span>pub struct </span>Rgb {
</span>    #[</span>arbitrary</span>(pattern = 0..=255)]
</span>    </span>r</span>: </span>u8</span>,
</span>    #[</span>arbitrary</span>(pattern = 0..=255)]
</span>    </span>g</span>: </span>u8</span>,
</span>    #[</span>arbitrary</span>(pattern = 0..=255)]
</span>    </span>b</span>: </span>u8</span>,
</span>}
</span></code></pre>
Automated testing strategies</h2>
Now that we've covered what the different ways of automated testing are, it's
worth briefly touching on some common testing strategies. Because once you've
understood that automated testing might be good, there often is a bit of a gap
to actually implement it in your own code. And that's what testing strategies
are for. These are some common testing strategies:</p>

roundtrip testing</strong>: generate a message, pass it to the encoder, then pass
the encoder's output to the decoder. The decoder's output should be the same
as the original message.</li>
differential testing</strong>: test the program against a "known good"
implementation of a similar program (also known as an "oracle"). This is
useful if, for example, you're re-implementing an existing protocol or algorithm.</li>
invariant testing</strong>: test that a certain property always</em> holds. This can
be a universal property like: "my program didn't crash". But invariants can be
specific too. Rain</a> recently gave a great example
that if you're for example testing a sorting function, you can check its
correctness by making iterating over each number and checking it's bigger than
the last number.</li>
</ul>
We've already covered roundtrip testing earlier in this post, and I believe
invariant testing should be fairly straight forward. So as we're closing out
here I just briefly wanted to show an example of how to perform differential
testing using an approach Tyler Neely</a> introduced me to a few years ago.</p>
The way this works is that you use your source of entropy to generate a sequence
of operations which you then apply to both your oracle and your program under
test. Should your test trigger a failure, the shrinker will kick in and try and
remove operations trying to uncover the minimal sequence of operations needed to
trigger the bug. Say we in the process of building the smallvec</a> crate; the
implementation differs from the stdlib, but the observable behavior should be
the same. Which means we can use the stdlib's Vec</code> as our oracle for the test,
which could look something like this:</p>
/// A single operation we can apply
</span>#[</span>derive</span>(Arbitrary)]
</span>enum </span>Operation {
</span>    Insert(</span>usize</span>, </span>usize</span>),
</span>    Fetch(</span>usize</span>),
</span>}
</span>
</span>#[</span>test</span>]
</span>fn </span>differential_smallvec_test</span>() {
</span>    heckcheck::check(|</span>operations</span>: Vec<Operation>| {
</span>        </span>// Setup both our subject and the oracle
</span>        </span>let mut</span> subject = smallvec::SmallVec::new();
</span>        </span>let mut</span> oracle = vec![];
</span>
</span>        </span>// Apply the same operations in-order to both the subject and the oracle
</span>        </span>// comparing outputs whenever we get any.
</span>        </span>for</span> operation in operations {
</span>            </span>match</span> operation {
</span>                Insert(index, value) => {
</span>                    oracle.</span>insert</span>(index, value);
</span>                    subject.</span>insert</span>(index, value);
</span>                }
</span>                Fetch(index) => {
</span>                    assert_eq!(oracle.</span>get</span>(index), subject.</span>get</span>(index));
</span>                }
</span>            }
</span>        }
</span>    });
</span>}
</span></code></pre>
Conclusion</h2>
In this post we've covered what fuzzing and property testing are, how they're
similar, and how they differ. We've introduced heckcheck</a>, a small
property-testing library and shown how you can use it to implement various
testing strategies with.</p>
I believe one of the greatest strengths Rust has is its ability to provide
uniform experiences. And I believe that automated testing is one of the best
tools we have to ensure the correctness of the implementations of our Rust
programs. I'd love to one day see Rust provide facilities out of the box for
automated testing - including a standard Arbitrary</code> interface.</p>

Tree-Structured Concurrency
Sat, 01 Jul 2023 00:00:00 +0000
For a while now I've been trying to find a good way to explain what structured
concurrency is, and how it applies to Rust. I've come up with zingers such as:
"Structured concurrency is structured programming as applied to concurrent
control-flow primitives"</em>. But that requires me to start explaining what
structured programming is, and suddenly I find myself 2000 words deep into a
concept which seems natural to most people writing programs today 1</a></sup>.</p>
Instead I want to try something different. In this post I want to provide you
with a practical introduction to structured concurrency. I will do my best to
explain what it is, why it's relevant, and how you can start applying it to your
rust projects today. Structured concurrency is a lens I use in almost all of my
reasoning about async Rust, and I think it might help others too. So let's dive in.</p>
This post assumes some familiarity with async Rust</a> and async cancellation</a>.
If you aren't already, it might be helpful to skim through the earlier posts on
the topic.</em></p>
^1</sup>If you're interested in structured programming though, I can
recommend Dijkstra's writing</a> on the subject. Most programming
languages we use today are structured, so reading about a time when when that
wasn't the case is really interesting.</p>
</div>
What is structured concurrency?</h2>
Structured concurrency is a property of your program. It's not just any
structure, the structure of the program is guaranteed to be a tree</a> regardless
of how much concurrency is going on internally 2</a></sup>. A good way to think
about it is that if you could plot the live call-graph of your program as a
series of relationships it would neatly form a tree. No cycles 3</a></sup>. No
dangling nodes. Just a single tree.</p>
^{2</sup>
When I was researching this topic last year I found a paper from I
believe the 80's which used the phrase "tree-structured concurrency". I couldn't
find it again in time for this post, but I remember tweeting about it because I
hadn't seen the term before and I thought it was really helpful!</p>
</div>}^{3</sup>
Yes yes, recursion can make functions call themselves - or even
call themselves through a proxy. By "live call-graph" I mean it like how flame
charts visualize function calls. Recursion is visualized by stacking function
calls on top of each other. The same idea would apply here by adding new async nodes
as children of existing nodes. The emphasis here is very much on live</em>
call-graph, not logical</em> call-graph.</p>
</div>}

        Fig 1. The arrows point from parent nodes to
        child nodes. It has no cycles. A parent can have multiple children. But
        a child always has a single parent - except for the root node.
    </figcaption>
</figure>
And this structure, at least in async Rust, provides three key properties:</p>

Cancellation propagation:</strong> When you drop a future to cancel it, it's
guaranteed that all futures underneath it are also cancelled.</li>
Error propagation:</strong> When an error is created somewhere down in the
call-graph, it can always be propagated up to the callers until there is a
caller who is ready to handle it.</li>
Ordering of operations:</strong> When a function returns, you know it is done doing
work. No surprises that things are still happening long after you thought the
function had completed.</li>
</ul>
These properties put together lead to something called a "black box model of
execution"</strong>: under a structured model of computing you don't need to know
anything about the inner workings of the functions you're calling, because their
behavior is guaranteed. A function will return when it's done, will cancel all
work when you ask it to, and you'll always receive an error if there is something
which needs handling. And as a result code under this model is composable</strong>.</p>

        Fig 2. Under structured concurrency every future has a parent,
        cancellation flows downward, and errors flow upward. When a future
        returns, you can be sure it's done doing work. 
    </figcaption>
</figure>
If your model of concurrency is unstructured</em>, then you don't have these
guarantees. So in order to guarantee that say, cancellation is correctly
propagated, you'll need to inspect the inner workings of every function you're
calling. Code under this model is not composable, and requires manual checks and
bespoke solutions. This is both labor-intensive and prone to errors.</p>
Unstructured concurrency: an example</h2>
Let's start by implementing a classic concurrency pattern: "race". But rather
than using structured</em> primitives, we can use the staples of unstructured
programming: the venerable task::spawn</code> and channel</code>. The way "race" works is
it takes two futures, and we try and get the output of whichever one completes
first is whose message we read. We could write it something like this:</p>
use </span>async_std::{channel, task};
</span>
</span>let </span>(sender0, receiver) = channel::bounded(</span>1</span>);
</span>
</span>let</span> sender1 = sender0.</span>clone</span>();
</span>task::spawn(async </span>move </span>{         </span>// 👈 Task "C"
</span>    task::sleep(Duration::from_millis(</span>100</span>));
</span>    sender1.</span>send</span>("</span>first</span>").await;
</span>});
</span>
</span>task::spawn(async </span>move </span>{         </span>// 👈 Task "B"
</span>    task::sleep(Duration::from_millis(</span>100</span>));
</span>    sender0.</span>send</span>("</span>second</span>").await;
</span>});
</span>
</span>let</span> msg = receiver.</span>recv</span>().await; </span>// 👈 Future "A"
</span>println!("</span>{msg}</span>");
</span></code></pre>
While this implements "race" semantics correctly, it doesn't handle
cancellation. If one of the branches completes, we'd ideally like to cancel the
other. And if the containing function is cancelled, both computations should be
cancelled. Because of how we've structured the program neither task is anchored
to a parent future, and so we can't cancel either computation directly. Instead
the solution would be to come up with some design using more channels, anchor
the handles - or we could instead rewrite this using structured primitives.</p>

        Fig 3. You can create a "race" operation by combining tasks and channels. Data
        can flow out of the tasks to the caller. But because the tasks aren't rooted
        in a parent task, cancellation doesn't propagate.
    </figcaption>
</figure>
Structured concurrency: an example</h2>
We can rewrite the example above using structured primitives instead. Rather
than DIY-ing our own "race" implementation using tasks and channels, we should
instead be using a "race" primitive which implements those semantics for us -
and correctly handles cancellation. Using the futures-concurrency</a> library we
could do that as follows:</p>
use </span>futures_concurrency::prelude::*;
</span>use </span>async_std::task;
</span>
</span>let</span> c = async {                   </span>// 👈 Future "C"
</span>    task::sleep(Duration::from_millis(</span>100</span>));
</span>    "</span>first</span>"
</span>};
</span>
</span>let</span> b = async {                   </span>// 👈 Future "B"
</span>    task::sleep(Duration::from_millis(</span>100</span>));
</span>    "</span>second</span>"
</span>};
</span>
</span>let</span> msg = (c, b).</span>race</span>().await;    </span>// 👈 Future "A"
</span>println!("</span>{msg}</span>");
</span></code></pre>
When one future completes here, the other future is cancelled. And should the
Race</code> future be dropped, then both futures are cancelled. Both futures have a
parent future when executing. Cancellation propagates downwards. And while there
are no errors in this example, if we were working with fallible operations then
early returns would cause the future to complete early - and errors would be
handled as expected.</p>

        Fig 4. By using a structured "race" primitive all child futures are rooted in
        a parent future. Which allows both cancellation and errors to propagate. And
        the operation won't return until all child futures have dropped.
    </figcaption>
</figure>
So far we've looked at just the "race" operation, which encodes: "Wait for the
first future to complete, then cancel the other"</em>. But other async concurrency
operations exist as well, such as:</p>

join</strong>: wait for all futures to complete.</li>
race_ok</strong>: wait for the first future to complete which returns Ok</code>.</li>
try_join</strong>: wait for all futures to complete, or return early if there is an error.</li>
merge</strong>: wait for all futures to complete, and yield items from a stream as
soon as they're ready.</li>
</ul>
There are a few more such as "zip", "unzip", and "chain" - as well as dynamic
concurrency primitives such as "task group", "fallible task group", and more.
The point is that the set of concurrency primitives</em> is bounded. But they can
be recombined in ways that makes it possible express any form of concurrency you
want. Not unlike how if a programming language supports branching, loops, and
function calls you can encode just about any control-flow logic you want,-
without ever needing to use "goto".</p>
What's the worst that can happen?</h2>
People sometimes ask: What's the worst that can happen when you don't have
structured concurrency? There are a number of bad outcomes possible, including
but not limited to: data loss, data corruption, and memory leaks.</p>
While Rust guards against data races</em> which fall under the category of "memory
safety", Rust can't protect you from logic bugs. For example: if you execute a
write</code> operation inside of a task whose handle isn't joined, then you'll need
to find some alternate mechanism to guarantee the ordering of that operation in
relation to the rest of the program. If you get that wrong you might
accidentally write to a closed resource and lose data. Or perform an
out-of-order write, and accidentally corrupt a resource 4</a></sup>. These kinds of bugs
are not in the same class as memory safety bugs. But they are nonetheless
serious, and they can be mitigated through principled API design.</p>
^{4</sup>
At a previous job we experienced exactly this in a database client: we
were having issues propagating cancellation correctly, which meant that the
connection protocol could be corrupted because we didn't flush messages when we
should have.</p>
</div>}Applying structured concurrency to your programs</h2>
task::spawn</strong></p>
When using or authoring async APIs in Rust, you should ask yourself the following questions to ensure structured concurrency:</p>

Cancellation propagation</strong>: If this future or function is dropped, will cancellation propagate to all child futures?</li>
Error propagation</strong>: If an error happens anywhere in this future, can we either handle it directly or surface it to the caller?</li>
Ordering of operations</strong>: When this function returns, will no more work continue to happen in the background?</li>
</ol>
If all of these properties are true, then once the function exits it's done
executing and you're good. This however leads us to a major issue in today's
async ecosystem: neither async-std nor tokio provide a spawn</code> function which is
structured.  If you drop a task handle the task isn't cancelled, but instead it's
detached and will continue to run in the background. This means that
cancellation doesn't automatically propagate across task boundaries, causing it
to be unstructured.</p>
The smol</a> library gets closer
though. It has a task implementation which gets us closer to "cancel on
drop"-semantics out of the box. Though it doesn't get us all the way yet because
it doesn't guarantee an ordering of operations. When a smol Task</code> is dropped
the task isn't guaranteed to have been cancelled, all it guarantees is that the
task will be cancelled at some point in the future.</p>
async drop</strong></p>
Which brings us to the biggest piece missing from async Rust's structured
concurrency story: the lack of async Drop in the language. Smol's
tasks have an async
cancel</a> method
which only resolves once the task has successfully been cancelled. Ideally we
could call this method in the destructor and wait for it. But in order to do
that today we'd need to block the thread, and that can lead to throughput
issues. No, in practice what we really need for this to work well is async
destructors 5</a></sup>.</p>
^5</sup>Async cancellation is hardly the only motivation for async Drop. It
also prevents us from encoding basic things like: "flush this operation on
drop"</a> -
which is something we can</em> encode in non-async Rust today.</p>
</div>
what can you do today?</strong></p>
But while we can't yet trivially fulfill all requirements for async structured
concurrency for async tasks, not all hope is lost. Without async Drop we can
already achieve 2/3 of the requirements for task spawning today. And if you're
using a runtime other than smol, adapting the spawn method</a> to work like
smol's does is not too much work. But most concurrency doesn't need tasks
because it isn't dynamic. For that you can take a look at the
futures-concurrency</a> library which implements composable primitives for
structured concurrency.</p>
If you want to adopt structured concurrency in your codebase today, you can
start by adopting it for non-task-based concurrency. And for task-based
concurrency you can adopt the smol model of task spawning to benefit from most
of the benefits of structured concurrency today. And eventually the hope is we
can add some form of async Drop to the language to close out the remaining
holes.</p>
Pattern: managed background tasks</h2>
People frequently ask how they can implement "background tasks" under structured
concurrency. This is used in scenarios such as an HTTP request handler which
also wants to submit a piece of telemetry. Rather than blocking sending the
response on the telemetry, it spawns a "background task" to submit the telemetry
in the background, and immediately returns from the request. This can look
something like this:</p>
let mut</span> app = tide::new();
</span>app.</span>at</span>("</span>/</span>").</span>post</span>(|_| async </span>move </span>{
</span>    task::spawn(async {  </span>// 👈 Spawns a background task…
</span>        </span>let</span> _res = </span>send_telemetry</span>(data, more_data).await;
</span>        </span>// … what if `res` is an `Err`? How should we handle errors here?
</span>    });
</span>    Ok("</span>hello world</span>")   </span>// 👈 …and returns immediately after.
</span>});
</span>app.</span>listen</span>("</span>127.0.0.1:8080</span>").await?;
</span></code></pre>
The phrase "background task" seems polite and unobtrusive. But from a structured
perspective it represents a computation without a parent - it is a dangling
task</em>. The core of the pattern we're dealing with is that we want to create a
computation which outlives the lifetime of the request handler. We can resolve
this by rather than creating a dangling task to submit it to a task queue or
task group which outlives the request handler.
Unlike a dangling task, a task queue</em> or task group</em> preserves structured
concurrency. Where a dangling task doesn't have a parent future and becomes
unreachable, using a task queue we transmit the ownership of a future to a
different object which outlives the current more ephemeral scope.</p>
I've heard people make the argument before that task::spawn</code> is perfectly
structured, as long as you think of it as spawning on some sort of unreachable,
global task pool. But the question shouldn't be whether tasks are spawned on a task
pool, but what the relationship is of those tasks to the rest of the program.
Because we cannot cancel and recreate an unreachable task pool. Nor can we
receive errors from this pool, or wait for all tasks in it to complete. It
doesn't provide the properties we want from structured concurrency - so we
shouldn't consider it structured.</p>
I don't feel like the ecosystem has any great solutions to this yet - in part
limited because we want "scoped
tasks"</a> which
basically require linear
destructors</a> to function.
But other experiments
exist</a> so we can use
that plus channels to put something together which gives us what we want:</p>
⚠️ Note: This code is not considered "good" by the author, and is merely used as an
example to show that this is possible to write today. More design work is
necessary to make this ergonomic ⚠️</em></p>
// Create a channel to send and receive futures over.
</span>let </span>(sender, receiver) = async_channel::unbounded();
</span>
</span>// Create a structured task group at the top-level, next to the HTTP server
</span>//
</span>// If any errors are returned by the spawned tasks, all active tasks are cancelled
</span>// and the error is returned by the handle.
</span>let</span> telemetry_handle = async_task_group::group(|</span>group</span>| async </span>move </span>{
</span>    </span>while let </span>Some(telemetry_future) = receiver.</span>next</span>().await {
</span>        group.</span>spawn</span>(async </span>move </span>{
</span>            telemetry_future.await?;  </span>// 👈 Propagate errors upwards
</span>            Ok(())
</span>        });
</span>    }
</span>    Ok(group)
</span>});
</span>
</span>// Create an application state for our HTTP server
</span>#[</span>derive</span>(Clone)]
</span>struct </span>State {
</span>    </span>sender</span>: async_channel::Sender<impl Future<Result<_>>>,
</span>}
</span>
</span>// Create the HTTP server
</span>let mut</span> app = tide::new();
</span>app.</span>at</span>("</span>/</span>").</span>post</span>(|</span>req</span>: Request<State>| async </span>move </span>{
</span>    state.sender.</span>send</span>(async {   </span>// 👈 Sends a future to the handler loop…
</span>        </span>send_telemetry</span>(data, more_data).await?;
</span>        Ok(())
</span>    }).await;
</span>    Ok("</span>hello world</span>")           </span>// 👈 …and returns immediately after.
</span>});
</span>
</span>// Concurrently execute both the HTTP server and the telemetry handler,
</span>// and if either one stops working the other stops too.
</span>(app.</span>listen</span>("</span>127.0.0.1:8080</span>"), telemetry_handle).</span>race</span>().await?;
</span></code></pre>
Like I said: we need to do a lot more API work to be able to rival the
convenience of just firing off a dangling task. But what we lack for in API
convenience, we make up for in semantics. Unlike our earlier example this will
correctly propagates cancellation and errors, and every executing future is
owned by a parent future. We could even take things a step further and implement
things like retry-handlers with error quotas on top of this to create a more
resilient system. But hopefully this is enough already to get the idea across of
what we could be doing with this.</p>
Guaranteeing Structure</h2>
I've been asking myself for a while now: "Would it be possible for Rust to
enforce structured concurrency in the language and libraries?"</em> I don't believe
this is something we guarantee from the language. But it is something can</em>
guarantee for Rust's library code, and make it so most async code is structured
by default.</p>
The reason why I don't believe it's fundamentally possible to guarantee
structure at the language level is because it's possible to express any kind of
program in Rust, which includes unstructured programs. Futures, channels, and
tasks as they exist today are all just regular library types. If we wanted to
enforce structure from the language, we would need to find a way to disallow the
creation of these libraries - and that seems impossible for a general-purpose
language 6</a></sup>.</p>
^{6</sup>
An example of this from structured programming:
Rust is a structured language. Assembly is not a structured language. You can
implement an assembly interpreter entirely in safe Rust - meaning you can
express unstructured code in a structured language. I could show examples of
this, but eh I hope the general line of reasoning makes sense here.</p>
</div>
Instead it seems more practical to me to adopt tree-structured concurrency as
the model we follow for async Rust. Not as a memory-safety guarantee, but as a
design discipline we apply across all of async Rust. APIs which are unstructured
should not be added to the stdlib. And our tooling should be aware that
unstructured code may exist, so it can flag it when it encounters it.</p>}Conclusion</h2>
In this post I've shown what (tree-)structured concurrency is, why it's
important for correctness, and how you can apply it in your programs. I hope
that by defining structured concurrency in terms of guarantees about propagation
of errors and cancellation, we can create a practical model for people to
reason about async Rust with.</p>
As recently reported by
Google</a>,
async Rust is one of the most difficult aspects of Rust to learn. It seems
likely that the lack of structure in async Rust code today did not help. In
async code today neither cancellation nor errors are guaranteed to propagate.
This means that if you want to reliably compose code, you need to have knowledge
of the inner workings of the code you're using. By adopting a (tree-)structured
model of concurrency these properties can instead be guaranteed from the outset,
which in turn would make Async Rust easier to reason about and teach. Because
"If it compiles it works"</em> should apply to async Rust too.</p>
Thanks to Iryna Shestak for illustrating and proof-reading this post.</em></p>

What is WASI?
Fri, 02 Jun 2023 00:00:00 +0000
Introduction</h2>
Now that the final touches are being put on the second version of
WASI</a> (WASI Preview 2), I figured now might be a good time to
write a few words on WASI, WebAssembly, and how to think about them.
In recent months I've been driving the WASI Preview 2 work in the Rust compiler.
So in order to do that I've had to familiarize where things are currently, where
things are headed, and how build a mental model for myself. Because while Wasm
and WASI have been around for a while, WASI especially has undergone a lot of
changes. Enough so that I figured it might be helpful to write a little about.</p>
What is WebAssembly?</h2>
WebAssembly</a> is a bytecode</em> format which compilers
can compile programs into. You can think of "bytecode" as an intermediate
format which can be converted to native</em> formats such as x86
assembly</a>, which is what
most desktop CPUs run, or ARM v8
assembly</a> which is what most mobile
phones run. This is useful because it means you can "compile once" to
WebAssembly Bytecode, and then run that code on any number of targets - provided
they have a runtime which can interpret WebAssembly and convert it to native
code.</p>
WebAssembly was initially designed for web browsers, to provide a way to execute
untrusted native code in a sandboxed environment. But this use case has since
expanded, because it turns out that a high-performance trusted sandbox is useful
abstraction for a lot of things - including creating sandboxed environments for
networked applications.</p>
The best way to think about the space WebAssembly runtimes occupy is roughly at
the same level as "docker containers" or "VMs". But rather than virtualizing an
entire operating system and managing that, WebAssembly runtimes use bytecode to
virtualize single applications. Where a single computer may concurrently execute
tens of VMs or hundreds of containers, it should be possible to run thousands of
WebAssembly programs 1</a></sup>.</p>
^{1</sup>
These are obviously not exact numbers, but more of a broad
illustration of scale. I know that for example FireCracker VMs are more
performant than regular VMs, and that people have successfully launched a million
plus docker containers in contests. But those are not typical cases, and I
believe in the typical</em> cases what I'm saying should be broadly accurate.</p>
</div>}bytecode, WebAssembly, and the JVM</h2>
WebAssembly is sometimes compared to Java and the
JVM</a>. This makes sense, because the
JVM is a popular platform which also interprets bytecode. The JVM not only
supports Java but any language which compiles to JVM Bytecode, including Kotlin
and Scala. However despite WebAssembly and the JVM both executing
bytecode, there are some key differences between the two:</p>

WebAssembly was designed to be targeted by native languages as C/C++/Rust in mind, while
the JVM is more oriented towards 2</a></sup> garbage-collected languages.</li>
WebAssembly is a royalty-free W3C
standard</a>, while the JVM
somewhat famously</a>
isn't. This means implementing WebAssembly support in language toolchains and
runtimes is not only possible, it's the exact goal of the project.</li>
WebAssembly was designed from the ground up with with strict sandboxing as a
core priority while the JVM wasn't. In order to enable secure multi-tenancy with
the JVM it can be advisable to wrap it in another isolation layer - such as a
VM.</li>
</ol>
^2</sup>This changes a bit with the introduction of
GraalVM</a>, but I think it still holds largely
true.</p>
</div>
Note though that I don't mean to harsh the JVM at all with this. WebAssembly and
the JVM were target very different use cases, have different priorities in their
design, and in turn excel at very different things.</p>
What are WebAssembly Components?</h2>
WebAssembly as bytecode format is also often referred to as "Core WebAssembly".
When you compile a "Core WebAssembly" program it is converted into something
called a "WebAssembly Module". You can roughly of think of this as an object
file in traditional compilation models. But unlike classic objects, Wasm modules
can't describe system calls or reference any external symbols - they can only
take numbers in and put numbers out, and that's about it. If you're trying to
make Wasm programs do anything other than sum up numbers, you need more than just Core WebAssembly.</p>
"WebAssembly Components" are a typed wrapper around Wasm Modules. Rather than
reasoning about numbers in/numbers out, they provide a way to talk about types,
functions, methods, and namespaces. The way this is done is via an IDL
format</a> called
WIT (Wasm Interface Types)
</a>.
Here's an example of a "monotonic clock" interface taken from the
preview2-prototyping
repo</a>:</p>
default interface </span>monotonic-clock {
</span>    </span>use poll</span>.</span>poll</span>.{</span>pollable</span>}
</span>
</span>    </span>/// A timestamp in nanoseconds.
</span>    </span>type instant </span>= </span>u64
</span>
</span>    </span>/// Read the current value of the clock.
</span>    </span>///
</span>    </span>/// The clock is monotonic, therefore calling this function repeatedly will
</span>    </span>/// produce a sequence of non-decreasing values.
</span>    </span>now</span>: func() -> instant
</span>
</span>    </span>/// Query the resolution of the clock.
</span>    </span>resolution</span>: func() -> instant
</span>
</span>    </span>/// Create a `pollable` which will resolve once the specified time has been
</span>    </span>/// reached.
</span>    </span>subscribe</span>: func(
</span>        </span>when</span>: instant,
</span>        </span>absolute</span>: bool
</span>    ) -> pollable
</span>}
</span></code></pre>
A Wasm Component is a single typed object consisting of a Core WASM Module plus
corresponding WIT definitions. Components can specify they either export</em>
specific interfaces, or require that other interfaces are imported</em>. For
example, I could write a binary program which prints the number of seconds
elapsed, which exports</em> a main function with no arguments, and imports</em> both
the stdout</code> and monotonic_clock</code> interfaces.</p>
One way to think about Wasm Components is: "What if we had ML-style modules in
our linker?", and that thought was taken all the way through from the system
call layer to the way libraries are linked to each other. With the added benefit
that each Wasm Component operates as its own security boundary, using a
"shared-nothing" approach to ensure isolation.</p>
What is WASI?</h2>
WASI stands for the "WebAssembly System Interface". Some people have recently
also started dubbing it the "WebAssembly Standard</em> Interfaces", since it
covers far more than just operating system interfaces. The way I think about
this is as sets of standard</em> Wasm Components which can be implemented by any
vendor and targeted by any toolchain. This can include APIs such as
socket-based networking, or filesystem access. But also APIs such as HTTP-based
clients and servers, or even message queue interfaces.</p>
Not all WASI interfaces are created equally though. For example: a serverless
environment may not want to expose direct access to the filesystem. Or the
stdlib of a programming language may not want to provide access to message queue
APIs. This differentiation in goals is why WASI has a notion of "Worlds". The
different "Worlds" are still in the process of being defined, but the
following two are actively being worked on right now:</p>

A world modeling a traditional operating system, providing access to sockets,
stdio, filesystem access, etc.</li>
A world modeling a "bursty" environment, typically best suited for serverless
applications. This will include access to handling HTTP requests, making HTTP
requests, access to object storage, message queues, etc.</li>
</ul>
The way WASI interfaces are standardized is via the W3C's WebAssembly working
group. People from across the industry come together to work on defining these
interfaces, which once accepted are then publish as a standard. This is
"standard" with a capital s. The standardization process can take time, but it
has the upside that once ratified you can know for a fact just about everyone in
the space will be adopting it.</p>
Categorizing WebAssembly</h2>
Now that we've talked about what Wasm and WASI are</em>, let's talk about what they
aren't</em>. Or well, perhaps more: what they aren't just</em>. WebAssembly falls into
multiple categories all at once, which means it also kind of escapes
categorization entirely. Depending on which angle you take, you can think of
WebAssembly as:</p>

A programming language</strong>: The WebAssembly text format is based on
S-Expressions</a>,
and it can be authored by hand if you want to. WIT is a similar format
targeted towards linkers, and incorporates a lot of work from the ML world.</li>
An operating system</strong>: Whether you're running WASI on Windows, on Linux, or
directly on hardware - the host you're targeting is WASI, and it should behave
the same everywhere. From a programmer's perspective, WASI becomes</em> the
operating system. The exact details of what's backing the runtime shouldn't
matter.</li>
A virtual machine</strong>: Wasm provides strict sandboxing guarantees, meaning that
you're able to trust a WebAssembly runtime with the same guarantees as you
would trust a VM. A Wasm program cannot ever jump out of its current memory, and
access memory of a different program running on the same computer. But it goes
one step further:</li>
An application runtime</strong>: Part of WASI are definitions for service-level APIs
such as "HTTP handling" and "message queue access". APIs which wouldn't be out
of place in an application runtime such as dotnet, or a cloud vendor such as Azure.</li>
A single-host container orchestrator</strong>: rather than dynamically linking
applications on the same host via HTTP, WASI provides an alternate model to
dynamically link programs without sharing any memory.</li>
A compiler</strong>: WebAssembly itself is a bytecode format which is the target of
language toolchains. But the Wasm runtimes themselves need to take that
bytecode and convert it to machine code, which means there is a lot of
compiler work involved</a>.</li>
An ABI, memory model, and linker</strong>: Wasm Components in many ways are</em> the
platform ABI. And because of how sandboxing has been implemented, one could
argue that Wasm has its own memory model. And the linker is basically
an ML module system in disguise. All put together, it's basically its own
model of a system, which behaves quite differently from most traditional
platforms.</li>
</ul>
In "Embrace the Kinda"</a>
SunfishCode talks about the categorical ambiguity of WebAssembly in more detail.</p>
Iterating on WASI</h2>
The latest version of WASI currently available is called: "WASI Preview 1". For
the past four years people have been hard at work on defining "WASI Preview 2",
and everything I've written about in this post has been about Preview 2. The
Preview 1 version of WASI was much closer to just</em> an operating system layer.
But it quickly became clear that this would hit some pretty big limitations, and
if WASI wanted to live up to its stated goals, it would need to change.</p>
Preview 2 is a complete rework of Preview 1, introducing WIT, WASM Components,
and all sorts of new standard interfaces. What it doesn't yet do is provide
first-class async primitives, which is scheduled for WASI Preview 3. Threads are
also missing from Preview 2, and work on that is still ongoing.</p>
One neat thing of the way WASI is structured is that virtualization layers can
be nested. In order to make upgrading between WASI versions easier, a shim, a
shim will be provided which allows Preview 1 code to continue working in hosts
which only implement Preview 2.</p>
Preview releases are intentionally backwards-incompatible, and the idea is for
them to be deprecated over time. Eventually the plan is to release a "WASI 1.0"
specification which will</em> provide more stability. The hope, at least, is that
with each WASI preview release, the number of changes between major versions
will shrink - so that the final 1.0 release will represent mostly a
formalization of what people have already been using for a while.</p>
The future of WASI</h2>
As mentioned, WASI Preview 2 doesn't yet have a model for first-class async, or
for threading. But it also doesn't yet provide any reasoning for multi-host
(distributed) linking of programs, nor are all of the WASI interfaces fully
specified yet. These are all things coming down the pipeline, which I'm really
excited about. Specifically for first-class async, the plan is to model that
using a structured model. Having access to that in</p>
Beyond core WASI features, there is a lot of other interesting work ongoing.
Maybe most interesting in my opinion is the paper: "Going beyond the Limits of
SFI: Flexible and Secure Hardware-Assisted In-Process Isolation with
HFI"</em></a> which was a joint project
between UCSD, Intel, and Fastly to provide a new set of instructions to make
in-process sandboxing faster and cheaper. This should greatly benefit
WebAssembly, allowing it to sandbox with even less overhead.</p>
Work on WASI Preview 2 is currently ongoing, and should be released later this
year. On behalf of Microsoft I'm currently working on the Preview 2
implementation for rustc together with folks from Fastly and Fermyon. But work
is simultaneously happening for other language toolchains, runtimes, and
platforms as well. With WASI Preview 2 including async socket support (though
not yet multi-threading), I think this may finally be the year WebAssembly is
finally going to start living up to its expectations. And I'm incredibly excited
for that!</p>
Conclusion</h2>
The way I roughly think about WebAssembly is as a hopeful vision of what
computing can be. It represents an opportunity to take the last 40 years of
operating system research, compiler development, and industry experience and
combines that into a form that is both coherent and accessible. I hope this post
can provide some insight on what WebAssembly and WASM are, and where things are
headed.</p>
Appendix A: Terminology Cheat Sheet</h2>

Wasm</strong>: short for "WebAssembly".</li>
WebAssembly</strong>, or "Core WebAssembly": A bytecode format which can be translated
to native code by a "WebAssembly interpreter" or "WebAssembly runtime"</li>
WASI</strong>: short for "WebAssembly System Interface". Sometimes also referred to as:
"WebAssembly Standard Interface". It provides a set of standardized component
interfaces using the WIT IDL language.</li>
WIT</strong>: short for "WebAssembly Interface Types" is a language to define
inter-component interfaces. It is used as part of WASM Components.</li>
IDL</strong>: "Interface Description Language" is a meta-language which defines program
interfaces. WIT is an example of an IDL.</li>
WASM Module</strong>: best thought of as "object files" for WebAssembly. It's a single
binary object containing core WASM bytecode.</li>
WebAssembly Component</strong>: a typed wrapper around a WASM Module. It combines a
typed WIT interface and WASM module to create a typed component.</li>
WebAssembly Worlds</strong>: A combination of different WASI interfaces, packaged
up into single environments. The brunt of development work in this space is
currently happening to define an "operating system world", and a "bursty
world" (serverless).</li>
WASI Preview 1</strong>: The snapshot of WASI development which was released in 2019.
This wasn't originally versioned and was previously just referred to as "WASI".</li>
WASI Preview 2</strong>: The version of WASI which is being releasing later this
year, built on top of WASM Components.</li>
Linker</strong>: A program which glues a number of other programs together into a
single program. The parts which are glued together are usually called
"libraries" or "objects", and the output program is usually called a "binary".
With WASM Components the differences between libraries and binaries are much
less clear.</li>
</ul>

Pattern Extensions
Fri, 26 May 2023 00:00:00 +0000
Microsoft Build is happening this week, and with it come new announcements for
the C# language and dotnet runtime. I was watching the video: "What's new in
C#12 and beyond"</a>, and the C#
team talked about pattern matching and pattern constructors. I've been thinking
a bit about patterns in Rust recently, and I wanted to share some of my doodles.</p>
Disclaimer time: I'm not on the Rust language team, nor do I hold any direct
decision making power. This post is not intended to be a complete or consistent
proposal, but rather to share some short ideas which seem neat. I'm not suggesting
we should prioritize this work over any other work, nor arguing that this should
be a high priority item. The reason I'm writing this is because I like to keep up with
programming languages - and C# made some cool announcements this week.</em></p>
Shapes of types</h2>
For the purpose of this post I want to distinguish between three different
shapes of types. The shapes are:</p>

List types, or "arrays" in Rust</li>
Sum types, or "enums" in Rust</li>
Product types, or "structs" in Rust</li>
String types, such as "hello"</code></li>
Ranges, such as 1..5</code></li>
</ul>
This is intentionally not a comprehensive list; neither on the side of the
shapes, nor on the side of the examples. For example: in Rust we also have
string-shaped items; and we also have vectors which are a kind of list. But this
is intentionally not a complete list - just enough of a categorization to allow
us to work through the remainder of this post.</p>
Pattern Initializers</h2>
In Rust we have different kinds of initializer syntax available today:</p>
let</span> x = [</span>0</span>; </span>12</span>];        </span>// Initialize an array containing twelve zeros
</span>let</span> f = Foo {};         </span>// Initialize a struct "Foo"
</span>let</span> b = Result::Ok(</span>12</span>); </span>// Initialize the enum `Result` to `Ok`
</span>let</span> r = </span>1</span>..</span>5</span>;           </span>// Initialize a range of one to five
</span>let</span> s = "</span>hello</span>";        </span>// Initialize a static str to "hello"
</span></code></pre>
Let's take the list-initializer syntax as an example. Say we wanted to
initialize not an array, but a vec. The way we would write that today would be
by doing:</p>
let</span> x = vec![</span>0</span>; </span>12</span>];
</span></code></pre>
The vec![]</code> macro is built into Rust, and imported by default so this works
for Vec</code>. But if we for example wanted to initialize a
VecDeque</code></a>
we'd be out of luck for the short-hand syntax, and we'd need to go through a
struct constructor instead.</p>
In C# 12 a new feature is added which is called "collection literals"
(demo</a>,
spec</a>)
which allows you to opt-in types to make use of constructor literal syntax. In
the demo they take an example of ImmutableList</code>, and allow it to be constructed
using the same syntax as a regular list. You can watch the demo or read the spec
to get a full picture of how this works in C#. But in Rust we could imagine
doing something very similar via a trait. Roughly sketching it out, you could
imagine this could work something like this:</p>
let</span> v = [</span>0</span>; </span>12</span>];               </span>// array
</span>let</span> v: Vec<_> = [</span>0</span>; </span>12</span>];       </span>// Vec
</span>let</span> v: VecDeque<_> = [</span>0</span>; </span>12</span>];  </span>// VecDeque
</span></code></pre>
Pattern Matching</h2>
Pattern matching is another interesting feature in Rust. In C# pattern matching
was added in (if I'm not mistaken) C# 6, and has seen steady improvements over
the past number of releases. In Rust we've had pattern matching since the very
beginning, but what we can match on has remained fairly limited to a number of
built-ins over time.</p>
Let's take a look again at the venerable list type in Rust: the array. We can
match on arrays by using the []</code> notation. Take for example the following,
whose first arm will match on the content of the array because it's the same:</p>
let</span> x = [</span>12</span>, </span>13</span>];
</span>match</span> x {
</span>    [</span>12</span>, </span>13</span>] => {}
</span>    _ => {}
</span>}
</span></code></pre>
For arrays that's all well and good. But for other list types it's more
complicated. Take for example the Vec</code> type: we can't directly match on it
using the same []</code> syntax. Instead we must obtain a slice</em> first via the
.as_slice</code> method or using the index syntax ([..]</code>):</p>
let</span> x = vec![</span>12</span>, </span>13</span>];
</span>match</span> x[..] {
</span>    [</span>12</span>, </span>13</span>] => {}
</span>    _ => {}
</span>}
</span></code></pre>
But while we have a workaround to enable matching for vec, we don't have that
option for many other container types. For example: VecDeque</code> doesn't
necessarily hold items in a single continguous slice of memory, so we can't
index into it directly using [..]</code> like we can with vec. The easiest way to do
this is to call VecDeque::make_contiguous</code></a>
to reorder items in memory and return a single unified slice. This does however
come with a runtime cost, which means we're now trading convenience for performance:</p>
let mut</span> x = VecDeque::new();
</span>x.</span>push_front</span>(</span>13</span>);
</span>x.</span>push_back</span>(</span>12</span>);
</span>match</span> x.</span>make_contiguous</span>() { </span>// this will re-allocate
</span>    [</span>12</span>, </span>13</span>] => {}
</span>    _ => {}
</span>}
</span></code></pre>
It would be far nicer if we could directly affect the matchers in Rust, rather
than needing to work via slices. A lot of interesting container types don't hold
memory in contiguous slices, so obtaining a slice can be difficult or expensive.
But it is often reasonably cheap to visit items in the container, which is all
we would need to enable pattern matching to work on richer types. Just like with
initializer patterns, it'd be nice if there was a way to enable pattern matching to be implementable via the trait system. What if this was possible instead?:</p>
let mut</span> x = VecDeque::new();
</span>x.</span>push_front</span>(</span>13</span>);
</span>x.</span>push_back</span>(</span>12</span>);
</span>match</span> x { </span>// no re-allocation because we match directly
</span>    [</span>12</span>, </span>13</span>] => {}
</span>    _ => {}
</span>}
</span></code></pre>
Pattern Types</h2>
Another interesting potential extension to Rust's pattern system is the notion
of "pattern types". In my series on state
machines</a> I've covered some of
this. But Oli pointed out that most of that actually falls under a feature
called "pattern types". This would allow us to not only reason about types as we
can today, but also about their refinements 1</a></sup>.</p>
^{1</sup>
I hope I'm using this word correctly here lol.</p>
</div>
Let's say we have a "double" function, which takes a number and doubles it. In
order for it to fit in the same number type, the number can't be more than half
as big. Because if we double it, we don't want to overflow the number type or use a bigger container. If we hand-coded this today, we'd probably want to write something like this:</p>}fn </span>u32_double</span>(</span>num</span>: </span>u32</span>) -> </span>u32 </span>{
</span>    </span>if</span> num >= </span>u32</span>::</span>MAX </span>/ </span>2 </span>{
</span>        panic!("</span>number cannot be doubled without overflowing</span>");
</span>    }
</span>    num + num
</span>}
</span>
</span>assert_eq!(</span>u32_double</span>(</span>1</span>), </span>2</span>); </span>// ✅ ok
</span>u32_double</span>(</span>u32</span>::</span>MAX</span>);         </span>// ❌ panic, numbers are too big
</span></code></pre>
This has an issue that this check exists entirely at runtime. If we initialize a
number literal such as in our passing example, we hope that the compiler's
optimizer kicks in and removes the check. But there is no guarantee that it
will. Instead using pattern types we can move this guarantee into the type
system, and we can omit the check in most cases. Using entirely made-up syntax,
you could imagine this could look something like this:</p>
// lol, idk what this should look like. let's just pretend ok?
</span>const</span> N: </span>u32 </span>= </span>u32</span>::</span>MAX </span>/ </span>2</span>;
</span>fn </span>u32_double</span>(</span>num</span>: </span>u32</span> is 0..N) -> </span>u32 </span>{
</span>    num + num
</span>}
</span>
</span>assert_eq!(</span>u32_double</span>(</span>1</span>), </span>2</span>); </span>// ✅ ok
</span>u32_double</span>(</span>u32</span>::</span>MAX</span>);         </span>// ❌ compile error: doesn't fit in the pattern range
</span></code></pre>
Pattern types would be nice for a number of other reasons too. As previously
mentioned, I believe they would make it easier to work with
the type state pattern in Rust. But I also believe they would allow to be more
specific about borrowing, enable libary authors to define their own NonZero</code>
types. But also provide us with what I believe may be the most ergonomic syntax
for try</code>-functions:</p>
// function notation
</span>try </span>fn </span>foo</span>() -> </span>u32</span> throws Err<io::Error> {..}  </span>// Result
</span>try </span>fn </span>foo</span>() -> </span>u32</span> throws None {..}            </span>// Option
</span>try </span>fn </span>foo</span>() -> </span>u32</span> throws Break<()> {..}       </span>// ControlFlow
</span>
</span>// closure notation
</span>try || {}                     </span>// infer everything
</span>try || throws None {}         </span>// infer success path
</span>try || -> </span>u32 </span>{}              </span>// infer error path
</span>try || -> </span>u32</span> throws None {}  </span>// infer nothing
</span></code></pre>
Tangent time: (it's my blog and you can't tell me what to do).
But what I like about this in particular is that it syntactically separates the
fallibiliy aspects of the function from the base of the function.
This is especially nice because for closure notations this allows us to provide
partial ascriptions. Most of the time you can probably just write try ||</code> and
the inference engine will help you out. But it's nice when you can write more of
the signature when you need to. Anyway, to close this out: I think pattern types
are pretty neat and they feel like they are a bit of a non-feature. In the sense
that they (at least to me) don't really change anything about the idea of Rust, they just
allow more code to compile which people reasonably could assume should already compile today.</p>
the "is" keyword</h2>
Rust recently (1.42, I know I know) stabilized the matches!</code> macro which allows
you to use patterns for comparisons. The way this works is as follows (taken from the docs):</p>
let</span> foo = '</span>f</span>';
</span>assert!(matches!(foo, '</span>A</span>'..='</span>Z</span>' | '</span>a</span>'..='</span>z</span>'));
</span>
</span>let</span> bar = Some(</span>4</span>);
</span>assert!(matches!(bar, Some(x) </span>if</span> x > </span>2</span>));
</span></code></pre>
It takes a variable on the left side, and a pattern on the right side, and then
compares the two. This is often nice to have when working within expressions
such as macros. Prior to the macro, it was common (and still is common) to add
is_</code> methods to enums, such as:
Result::is_ok</code></a>
to allow easy comparisons within expressions.
In C# (and other languages too) there exists the is</code>
keyword</a>
which elevates this to a language feature. With it we could rewrite the previous
example as:</p>
let</span> foo = '</span>f</span>';
</span>assert!(foo is '</span>A</span>'..='</span>Z</span>' | '</span>a</span>'..='</span>z</span>');
</span>
</span>let</span> bar = Some(</span>4</span>);
</span>assert!(bar is Some(x) </span>if</span> x > </span>2</span>);
</span></code></pre>
Whether this feature would carry its weight is a good question. Right now it's
good practice to add is_</code> methods for each enum member, which is a lot of typing. So much so that I added a
generator to Rust-Analyzer for
it</a> to automate the process. If every member in
every enum would benefit from a corresponding method, maybe having it as a
language feature would not be a bad idea? 2</a></sup></p>
^{2</sup>
Tangent: I'm still wondering about a different timeline where the as</code>
keyword would make use of Into</code> instead of doing primitive casts. And as?</code>
could be mapped to TryInto</code>. I don't know enough about the topic to say
anything about why that isn't the case / couldnt't be made to work. But conceptually I kind of like the idea of having both as</code> and is</code> as a pair of keywords to operate on types.</p>
</div>}On Product Patterns</h2>
At the start of this post I mentioned there are a number of shapes types can be,
but so far we've only looked at list types. I've been looking at C# and while
support for matching on product types (C#
Records</a>)
seems to roughly match to what Rust is doing - I'm less sure about what the
story will be for constructing or matching on e.g. hashmap structures. It seems
the following syntax (collection
initializers</a>)
is supported today to construct hashmaps:</p>
var </span>data </span>= new Dictionary<</span>string</span>, </span>string</span>> {
</span>    { "</span>test</span>", "</span>val</span>" }, 
</span>    { "</span>test2</span>", "</span>val2</span>" }
</span>};
</span></code></pre>
This isn't quite the same as collection literals</em>, and I also don't know what
that the future plans for pattern matching are in C#. Also I'm kind of coming up
empty for other languages on how e.g. matching on arbitrary product types should
look like. That's a long way of saying: for any record types that aren't just
structs, I'm unsure what the syntax should be. But because we're fun-posting,
here's three potential options that I'm somewhat certain are all not a good
idea:</p>
// Ehhhh option 1: reinterpret struct tokens as
</span>// values. This seems really weird and bad?
</span>// Downside: record names are not string literals,
</span>// and string literals are not the only possible keys.
</span>let</span> x: HashMap<&</span>'static str</span>, </span>u32</span>> = {
</span>    hello: </span>12</span>,
</span>    world: </span>13</span>,
</span>};
</span>match</span> x {
</span>    { hello: </span>12</span>, world: </span>13 </span>} => {}
</span>    _ => {}
</span>};
</span>
</span>// Ehhhh option 2: Literals in both the key and the value
</span>// position kind of like JS Object notation.
</span>// Downside: this syntax is ambiguous, because it's 
</span>// the same as block scope syntax.
</span>let</span> x: HashMap<&</span>'static str</span>, </span>u32</span>> = {
</span>    "</span>hello</span>": </span>12</span>,
</span>    "</span>world</span>": </span>13</span>,
</span>};
</span>match</span> x {
</span>    { "</span>hello</span>": </span>12</span>, "</span>world</span>": </span>13 </span>} => {}
</span>    _ => {}
</span>};
</span>
</span>// Ehhhh option 3: The only initializer syntax is [],
</span>// so we tuple our way through it.
</span>// Downside: this implies ordering where there is none.
</span>let</span> x: HashMap<&</span>'static str</span>, </span>u32</span>> = [
</span>    ("</span>hello</span>", </span>12</span>),
</span>    ("</span>world</span>", </span>13</span>),
</span>];
</span>match</span> x {
</span>    [("</span>hello</span>", </span>12</span>), ("</span>world</span>", </span>13</span>)] => {}
</span>    _ => {}
</span>};
</span></code></pre>
Anyway tldr: where pattern initializers and matchers for list-like types seem
like a relatively straighforward design, a syntax for product types (maps,
sets) seems much less obvious to me.</p>
On String Patterns</h2>
In Rust 2021 we already reserved token
prefixes</a>
with the intent of allowing new string literals such as f"hello {world}"</code> for
format strings or s""</code> for allocated strings. So we're already somewhat on the way
to expand initializer syntax. I think it would make sense if we could work in
the inverse and as we work on that also allow the inverse to work:</p>
// Rust today
</span>match </span>String::from("</span>hello</span>") {
</span>    "</span>hello</span>" => {}   </span>// ❌ Compile error: expected `String`, found `&str`
</span>    _ => {}
</span>}
</span>
</span>// What if something like this could work instead?
</span>// (string literals + string matching)
</span>match</span> s"</span>hello</span>" {    </span>// String literal constructor
</span>    "</span>hello</span>" => {}   </span>// String pattern matching (without casting to string slice)
</span>    _ => {}
</span>}
</span></code></pre>
Conclusion</h2>
As I mentioned at the start, this is a bit of a rambly/braindumpy post because I
saw some cool C# things today and I wanted to write about that. I think it's fun
to think through what it could look like if Rust's pattern system became more
capable. To briefly recap what we've shown about pattern initializers + matchers:</p>
// Creating + matching on a `VecDeque` today.
</span>let mut</span> x = VecDeque::new();
</span>x.</span>push_front</span>(</span>13</span>);
</span>x.</span>push_back</span>(</span>12</span>);
</span>match</span> x.</span>make_contiguous</span>() {
</span>    [</span>12</span>, </span>13</span>] => {}
</span>    _ => {}
</span>}
</span>
</span>// Creating + matching on a `VecDeque` if we had
</span>// implementable list initializers + list patterns
</span>let mut</span> x: VecDeque = [</span>12</span>, </span>13</span>];
</span>match</span> x {
</span>    [</span>12</span>, </span>13</span>] => {}
</span>    _ => {}
</span>}
</span></code></pre>
That's all. ✌️</p>

Linear Types One-Pager
Tue, 28 Mar 2023 00:00:00 +0000
This post represents an overview of an MVP "linear types" design which we could
probably start implementing and validating today</em> if we wanted to. What I'm
sharing here is a combination of conversations I've had with
Gankra</a> and Jonas
Sheevink</a>.</p>
Prior Reading</h2>

The Pain Of Real Linear Types in Rust</a></li>
Must move types</a></li>
Linearity and Control</a></li>
</ul>
What are linear types?</h2>
We typically frame linear types as: "Real #[must_use]</code> types"</em>.  But it seems
like what we should be doing is framing linear types as: "Types which guarantee
their destructor will be called."</em> This is enough to provide the benefit we
feature we actually want: a guarantee that destructors will be called, so we can
rely on them for soundness purposes.</p>
A minimal implementation</h2>
The estimate is that something like this would probably take a skilled compiler
engineer on the order of days, not weeks or months, to validate:</p>

We define a new unsafe</code> (auto-)trait named Leak</code>/Leave</code>/Forget</code></li>
All bounds take an implicit + Leak</code> bound, like we do for + Sized</code>.</li>
Certain functions such as mem::forget</code> will always keep taking + Leak</code> bounds.</li>
Functions which want to opt-in to linearity can take + ?Leak</code> bounds.</li>
Types which want to opt-in to linearity can implement !Leak</code> or put a
PhantomLeak</code> type in a field.</li>
</ul>
This wouldn't require any special type of Drop</code> either, we just guarantee it
will always be called for all !Leak</code> types. I have a feeling this
design is actually really close to what Niko was describing in his post. The
main difference is that we consider a destructor being run as enough to satisfy
the "must use" semantics of linear types. So the only difference between linear
and affine types is that linear types can't be safely passed to APIs such as
mem::forget</code> and Arc::new</code>.</p>
Interactions with control flow</h2>
The only way to prevent Drop</code> from running is if we pass it to mem::forget</code>,
create a cycle using Rc</code>, Arc</code>, or using static</code>. The first relies on the ManuallyDrop</code>
built-in, the second relies on the UnsafeCell</code> built-in, and the last is a
language item. We can disallow the use of !Leak</code> types in static</code> lang items.
And for the functions we can make it so the bounds will always need to take + Leak</code>, meaning all of their derivatives will too. If a function wants to use
take ?Leak</code> as a bound and pass it to + Leak</code> types, the only way to do that
is via an unsafe cast. That would mean it is on the hook for upholding the
safety invariants of the ?Leak</code> type. All other interactions with e.g.
panic!</code>, .await</code>, or ?</code> would Just Work as expected and we don't need to do
anything for them.</p>
If people want to put a !Leak</code> type in an Arc</code>, they can create an unsafe
wrapper which temporarily removes the !Leak</code> bound from the type, and put that
in an Arc</code>. They'd then be manually on the hook for ensuring destructors are
run, but that's okay.</p>
For async types, we don't yet have a async Drop</code> design. But !Leak</code> types
would guarantee that async Drop</code> is always run. This would be enough to
guarantee the "must consume" semantics we want for certain types.</p>
Drawbacks and Challenges</h2>
This scheme would work and could be implemented in record time. In my opinion we
should do this on nightly, just to prove that it can be done. Once done we can
tackle the ergonomics issues this creates:</p>

We would want to go through the entire stdlib and mark almost every
generic param as + ?Leak</code>.</li>
Because trait bounds are instantly stable, we would have no trial period to
test out a linear bound, before committing to it.</li>
Just like how ?Sized</code> describes "maybe-dyn", ?Leak</code> describes
"maybe-must-move". Because we're talking in terms of negation, it is really
hard to reason about. Our dyn system is notorious for being some of the hardest
to understand part of Rust.</li>
Assuming all but a few bounds in the stdlib will eventually take ?Leak</code>, will
we ever want to switch to make that the default across an edition to reduce line
noise? If we think we might, we should think about what that would look like.</li>
Leak</code> introduces new safety rules for Rust which must be upheld when
declaring ?Leak</code> or !Leak</code> bounds. Those will need to be spelled out in detail
before Leak</code> can be stabilized.</li>
</ul>
I believe that just like with dyn</code>, "must not move", send, etc we should look
at an alternate formulation of these bounds by treating them as built-in
effects. That would allow us to address the issues of versioning, visual noise,
etc. in a more consistent and ergonomic way. But that's not a requirement to
start testing this out, so if we believe we want this feature, starting with the
design in this document seems to me like the right way to go.</p>
Updates</h2>
update 2023-05-03:</strong> Jack Rickard</a> pointed out that !Leak</code> types and statics</a>
have bad interactions. Types which are placed in statics don't have their
destructors run when the program exits. Which means that if we allowed !Leak</code>
types to be placed inside statics, it would prevent destructors from running
entirely. This would break the linearity guarantees. so statics have to be a
no-no for linear types. This puts linearity right next to const in that statics
are fundamentally disallowed.</p>
2023-05-04:</strong> From talking more to Jack Rickard, we created another example of
"safe forget" using thread::spawn</code>:</p>
fn </span>safe_forget</span><T>(</span>val</span>: T) {
</span>    thread::spawn(</span>move </span>|| {
</span>        </span>let</span> val = val;                   </span>// move `val` into the thread
</span>        </span>loop </span>{ std::thread::park(); }    </span>// park the thread indefinitely
</span>    });                                  </span>// drop the thread thandle
</span>}
</span></code></pre>
The thread handle becomes unreachable, detaching the thread, and holding the
value until the main thread exits - at which point destructors won't run
1</a></sup>. This is contrast to thread::scope</code>, which is
tree-structured and wouldn't allow doing this. To me this seems to indicate
three things:</p>

In order to guarantee destructors are run, it's key to prevent cycles. This
holds for both data (Rc</code>/Arc</code>) and computation (thread::spawn</code>). I've got
a hunch that structured concurrency is very closely connected to what we're
trying to do here.</li>
If we are to introduce linearity into Rust, we need to follow the path we've
taken with const</code>. One API at the time, validating the linearity guarantees
at each stage. This emphasizes the need to have a formalization of the safety
rules.</li>
Even in this model we need to have a way to express ?Leak Fn</code> to enable
closing over !Leak</code> types. This adds further credibility to the idea that
we will want some form of surface syntax for !leak fn</code>, to be able to
declare free functions which can be passed into linear closure position. This
is in addition to the existing !leak async fn</code> use case.</li>
</ol>
^{1</sup>
It makes me wonder about an alternate timeline where
JoinHandle</code> would be !Leak</code> and implement "join on drop" semantics. Had that
been the case, then thread::spawn</code> would probably have been okay to close over
!Leak</code> types.</p>
</div>}

Linearity and Control
Thu, 23 Mar 2023 00:00:00 +0000
Introduction</h2>
Update 2023-03-28</strong>: I've published a follow-up to this
post</a> which presents an
alternate design, reframing linearity as: "Drop is guaranteed to run". To get a
full overview of the design space, read this post to get a sense of the problem
domain, and the follow-up post for a concrete solution.</em></p>

A week ago Niko published a post on linear types</a>, introducing
the idea of "must move" types, which he suggested could be implemented through
some form of ?Drop</code> bound. It's far from the first time linear types have come
up. Five years ago Gankra also published a post</a> on linear types,
explaining what they are and why they're hard to get right.</p>
In this post I want to build on these two posts; expanding on what linear types
are, why they're useful, how they would interact with Rust, and share a novel
effect-based design — which unlike many previous attempts would preserve our
ability to implement and use destructors.</p>
Before we dive in I want to extend an enormous thanks to Eric Holk, for first
positing the idea that linearity obligations might be transferable up the call
stack or ownership chain. That has been the key insight on which the majority of
the design in this post builds.</em></p>
⚠️ Disclaimer: The syntax in this post is entirely made up, and meant as
illustrative only. While it may be tempting to discuss syntax, this post is
primarily focused on semantics instead. I am not a member of the language team.
I do not speak for the language team. This post does not encode any decisions
made by the language team. The point of this post is to exchange ideas, in the
open, with peers - because in my opinion that is the best way to go about
designing things.️</em></p>
What are linear types?</h2>
If you've been writing Rust for a little while, you'll be familiar with
#[must_use]</code>. Decorating a function with this attribute makes it so the types
returned from the functions must be used</em> or else it produces a warning.
But #[must_use]</code> is limited. "using" in this context means "doing literally
anything with the function output". As long as we do that, #[must_use]</code> is
happy. This includes, or example, passing it to mem::forget</code> or Box::leak</code>.</p>
must_use_fn</span>();              </span>// ⚠️ "unused return value of `must_use_fn` that must be used"
</span>let</span> x = </span>must_use_fn</span>();      </span>// ✅ no warnings or errors
</span>let </span>_ = </span>must_use_fn</span>();      </span>// ✅ no warnings or errors
</span>mem::forget(</span>must_use_fn</span>()); </span>// ✅ no warnings or errors
</span></code></pre>
However there are often cases where we would like a more rigorous version of
#[must_use]</code>. Say we have a Transaction</code> type. It has a method new</code> to create
a transaction, a method commit</code> to finalize the transaction successfully, and a
method abort</code> to finalize the transaction unsuccessfully.</p>
/// A type representing a transaction.
</span>struct </span>Transaction { .. }
</span>impl </span>Transaction {
</span>    </span>/// Create a new instance of `Transaction`.
</span>    </span>fn </span>new</span>() -> </span>Self </span>{ .. }
</span>    </span>/// Finalize `Transaction` successfully.
</span>    </span>fn </span>commit</span>(</span>self</span>) { .. }
</span>    </span>/// Finalize `Transaction` unsuccessfully.
</span>    </span>fn </span>abort</span>(</span>self</span>) { .. }
</span>}
</span></code></pre>
What we'd like to express here is that if you call new</code>, you must</em> call either
commit</code>, or abort</code>. But Rust's type system doesn't let us express this yet.
Even Drop</code> wouldn't help us here, since the type system doesn't guarantee it'll
ever be called. What we'd like is for the following to be possible:</p>
let </span>_ = Transaction::new(); </span>// ❌ "unused return value of `Transaction::new` that must be used"
</span></code></pre>
The reason for this is that Rust type system supports affine</em> types, but does
not support linear</em> types. This is a common misconception, but the difference
between them is actually pretty important:</p>

affine types</strong> are types which must can be used at most once</em>.</li>
linear types</strong> are types which must be used exactly once</em>.</li>
</ul>
This means that Rust can guarantee you won't use the same type twice (e.g. we systemically prevent
double
free</a>s from happening);
but for example we can't guarantee that values will</em> have methods called on
them (e.g. we can't prevent memory from leaking). Being able to express
linearity in Rust would likely also enable us to solve some of the other
challenges we've been facing:</p>

async scopes</strong>: thread::scope</code></a> allows us to share variables between
threads without using locks. An async/.await</code> version of this would
require linear types to function 1</a></sup>.</li>
true session types</strong>: This is best explained as: "What if we could model the
semantics of entire network protocols using the type system." We can get close
today, but to fully encode session types we require access to linear types
[^session-type-limits]  2</a></sup>.</li>
more efficient completion-based IO</strong>: APIs such as io_uring</code> have very specific
rules about how objects should be passed between the kernel and the program.
Linear types should make it possible to model these interactions with
significantly less runtime overhead 3</a></sup>.</li>
more efficient async C++ FFI</strong>: The Future</code>-equivalent in C++ must</em> be
polled to completion because if you don't you trigger UB. This is unlike
Rust's Future</code> type which can be dropped to be cancelled. Linear types would
allow us to model this behavior directly in Rust's type system.</li>
</ul>
^1</sup>Tmandry wrote a good post about the challenges of async
scopes</a>. While he
doesn't directly spell out how linear types would help solve some of these
issues Niko
does</a>.</p>
</div>
^4</sup>The paper "Session types for
Rust"</a> writes in their
conclusion: "We have demonstrated that session types can be implemented
directly in a language supporting affine types, and argued for its safety. Like
linear types, affine types prevent aliasing, but, unlike linear types, fail to
promise progress."</em></p>
</div>
^2</sup>There's a cool
paper</a>
and talk</a> titled "Propositions as
Types" by Phil Wadler who makes the point that every good idea is
discovered</em>, not invented. In the talk at the 37:00 mark he mentions that
linear logic and session types seem to correspond to each other, and may in fact
provide us with a solution to way to formally encode concurrency and
distribution. Whether that's actually true, I don't know for sure - I'm not a
professor in theoretical computer science like Phil is, nor have I ever used
linear types or session types to say anything from experience. But it certainly
seems plausible, which I think should be enough to get anyone excited who's
trying to build reliable networked services.</p>
</div>
^3</sup>Tbh I'm not an expert on io_uring</code>, so I might not be 100% on
point here. I'm basing my understanding of this mostly on the fact that the
rio</code></a> library would be considered sound
if the destructors could be guaranteed to run. The point of linear types is to
provide such a guarantee, which means linearity could be a way to model
completion-based IO safely and ergonomically, directly from Rust.</p>
</div>
To provide a quick example of something which is inexpressible in Rust today: we
can't create an async/.await</code> version of
thread::scope</code></a> in Rust
today. But using linear types, we could require that the future returned by
async_scope</code> is awaited, and that in the event of a cancellation, the
destructors of async_scope</code> are guaranteed to run for as long as tasks are
live, ensuring we maintain Rust's property of preventing all data races in safe
Rust 5</a></sup>.</p>
^5</sup>This is a very similar problem Gankra described in here
2015 post on why why mem::forget</code> should be
safe</a>. It caused us to remove
thread::scope</code> from the stdlib until its reintroduction in 2022. Unfortunately
the tricks we applied to make this work in stable Rust today won't work for
async Rust; which means we have to solve this issue for real this time</em> if we
want it to be possible.</p>
</div>
let mut</span> a = vec![</span>1</span>, </span>2</span>, </span>3</span>];
</span>let mut</span> x = </span>0</span>;
</span>
</span>async_scope</span>(|</span>s</span>| {
</span>    s.</span>spawn</span>(|| {
</span>        println!("</span>hello from the first scoped task</span>");
</span>        </span>// We can borrow `a` here.
</span>        dbg!(&a);
</span>    });
</span>    s.</span>spawn</span>(|| {
</span>        println!("</span>hello from the second scoped task</span>");
</span>        </span>// We can even mutably borrow `x` here,
</span>        </span>// because no other tasks are using it.
</span>        x += a[</span>0</span>] + a[</span>2</span>];
</span>    });
</span>    println!("</span>hello from the main task</span>");
</span>}).await;
</span>
</span>// After the scope, we can modify and access our variables again:
</span>a.</span>push</span>(</span>4</span>);
</span>assert_eq!(x, a.</span>len</span>());
</span></code></pre>
Create and consume</h2>
Okay, time to get into the details of the design. At the heart of a linear
type's lifecycle are two phases: create (or "construct") and consume (or
"destruct"). A linear type promises at the type system level that once it's
created, it will always</em> be consumed. The way we're modeling this in our design
is using type instantiation to create and pattern destructuring</a> to consume</p>
^{6</sup></div>}^6</sup>This example was inspired by Firstyear's post on Transactional
Operations in
Rust</a>.</p>
</div>
let</span> txn = Transaction { .. };  </span>// 1. Create a new instance of `Transaction`.
</span>let</span> Transaction { .. } = txn;  </span>// 2. Consume an instance of `Transaction`.
</span></code></pre>
No matter what happens in the code, every linear type which is created
will always have a matching consume. This means that as long as your program
has an opportunity to run to completion it is guaranteed</em> to be consumed. Yay
for type systems!</p>
In this post we'll be modeling this via the introduction of a new effect kind
(or "modifier keyword" if you prefer): linear</code>. Don't worry too much about the
name, I'm mostly using it for clarity throughout this post. This effect can be
applied to either types or functions; but we'll start with types.</p>
Linear types</h3>
When a type is marked as linear</code> it means that once a type has been
created, in order for it to type-check it must also be consumed. This will
be automatically done by the type checker for you, who will gently tell you if
you messed up anywhere. But by using unsafe</code> you can also do this by hand if
you need to. More on that later though.</p>
linear </span>struct </span>Transaction { .. }
</span>linear </span>enum </span>Foo { .. }
</span></code></pre>
This is not the end of linear types though: a key feature of types is that they
can be composed into new types. When a type takes a linear type as an argument,
the "linearity" obligation transfers to the enclosing type. This means that a
linear type can only be held by another linear type:</p>
linear </span>struct </span>TransactionWrapper {
</span>    </span>txn</span>: Transaction, </span>// linear type
</span>}
</span></code></pre>
Linear functions</h3>
The second place where the linear</code> notation can be used is on functions and
methods. The way this would look is like this:</p>
impl </span>Transaction {
</span>    </span>fn </span>new</span>() -> </span>Self </span>{ .. }
</span>    linear </span>fn </span>commit</span>(linear </span>self</span>) { .. }
</span>    linear </span>fn </span>abort</span>(linear </span>self</span>) { .. }
</span>}
</span></code></pre>
Functions which don't take any linear types as arguments don't have to be marked as
linear fn</code> which is why fn new</code> is not marked linear here 7</a></sup>. But if a function wants
to take a linear type as an argument, it needs to be marked as a linear fn</code>.
Every linear function needs to promise it will do one of three things for every
instance of a linear type which is passed to it:</p>
^{7</sup>
Folks in review asked me: "why doesn't fn new</code> need to be
linear?" The real reason is because it doesn't follow the rules I'm setting out
in this post. But it could very well be that it would me a lot more clear if
only linear functions could yield linear types, in which case this should be
linear fn new</code>. As I've said at the start of this post: syntax is something we
can dig into once we know the semantics we want to expose. And answering this
question really is just a matter of syntax.</p>
</div>}
It will destruct the instance.</li>
It will yield the instance to the caller along with control over execution.</li>
It will call another linear function and pass the instance along as an argument.</li>
</ol>
This might sound easy enough, but in practice this has implications for what
can't</em> be done, such as:</p>

It will not mem::forget</code> any instances of linear types.</li>
It will not create any cycles around instances of linear types using Arc/Rc</code> or similar.</li>
</ol>
If we can imagine a parallel version of the stdlib where all types and functions
which can be linear are</em> in fact linear: mem::forget</code>, Arc::new</code>, and
Rc::new</code> would not exist. And our safety rules would disallow creating new
linear fn</code>s which don't uphold those rules. Being able to mark which functions
uphold the rules of linearity is exactly the purpose of the linear fn</code>
notation.</p>
Linear contexts</h3>
The "must consume" obligation of a linear function only applies to arguments
which themselves are also linear. It should be possible to pass non-linear
arguments to linear functions without any problem, allowing you to mix and match
linear and non-linear types within a linear function.</p>
Similarly: it should be possible to create linear types and call linear
functions from non-linear functions. This means you should be able to create
and consume linear types from your existing functions largely without problems.
The exceptions here are probably async</code> contexts, and things like closure
captures: both of those create types which wrap existing types. So when they
capture a linear type, they themselves must become linear too. We'll get into
more detail on how that works later on in this post.</p>
This means that it should be possible to for example have an fn main</code> which is
not marked linear</code> which creates and consumes linear types in its function
body:</p>
fn </span>main</span>() {                        </span>// 1. Non-linear function context.
</span>    </span>let</span> txn = Transaction { .. };  </span>// 2. Create a new instance of `Transaction`.
</span>    </span>let</span> Transaction { .. } = txn;  </span>// 3. Consume an instance of `Transaction`.
</span>}
</span></code></pre>
If for whatever reason we do find a good reason why linear types should only be
accessible from linear contexts, we could do that. Just like with const {}</code> we
could create a linear {}</code> context which can be used to close out the linearity
effect. But I don't think that is necessary, and I suspect linear types would be
easier to use if we didn't require this. This is almost a question of syntax
though, so it's probably not worth going too deep into this right now.</p>
Linear methods</h3>
I've sort of hinted at this, but not said it out loud: for linear types to
be really useful it's not enough if we have types which cannot be dropped. What
we really want is pairs of methods which need to be called.</p>
In Rust we don't have "class constructors" built into the language as a feature.
Instead what we have is struct initialization, and methods such as
MyStruct::new</code> or MyStruct::open</code> to construct the type. The reason why these
work is because they're the only places in the code which have access to the
struct's private fields. So to create the struct, they're the only way to do it.</p>
impl </span>Transaction {
</span>    </span>pub fn </span>new</span>() -> </span>Self </span>{ 
</span>        </span>Self </span>{ field1, field2 }
</span>    }
</span>}
</span></code></pre>
Similarly with linear types what we don't want is for people to require access to
private fields to destruct their types. Instead we can just create methods which
can do the destruction for us. This means that the only way to close out the
linear</code> effect's requirements is to call a destructor method on the type:</p>
impl </span>Transaction {
</span>    </span>pub</span> linear </span>fn </span>commit</span>(</span>self</span>) {
</span>        </span>let Self </span>{ field1, field2 } = </span>self</span>;
</span>    }
</span>    </span>pub</span> linear </span>fn </span>abort</span>(</span>self</span>) { 
</span>        </span>let Self </span>{ field1, field2 } = </span>self</span>;
</span>    }
</span>}
</span></code></pre>
As we discussed earlier: the only valid thing to do with an instance of a linear
type is to either consume it, return it, or pass it to another linear function.
"linear destructors" do the latter: they're linear functions which consume a
linear type. So from a compiler's perspective this is exactly right.</p>
Linear drop</h3>
A key feature feature in Rust is the ability to define Drop</code> destructors
(RAII</a>).
For example when you create a file using File::open</code>, its Drop</code> destructor
will make sure that the file is closed when it goes out of scope. Syntactically
this makes code a lot nicer to read, because rather than needing to remember
which function to call inside a function body, the type implementing Drop</code> will
just auto-cleanup all resources for you.</p>
To ensure "linear Rust" is as ergonomic as non-linear Rust, code should largely
feel the same as well. This includes having access to Drop</code>. The way we could
do this is by introducing a linear version of Drop</code>, which rather than take
&mut self</code> takes self</code>. And when called requires the existing rules of
linearity are upheld. The way this destructor would work is that it would only</em>
run when a type goes out of scope. It would not</em> run if a type is manually
destructed.</p>
People may be wondering at this point what the difference between linear types
and affine types then is, if both get access to destructors. With affine types
we can't guarantee that destructors will run, so we can't build on it for the
purpose of soundness. With linear types we can make that guarantee, so we
destructors can be relied on for the purpose of soundness.</p>
In our Transaction</code> example, we can imagine that we might to abort the
transaction unsuccessfully if it goes out of scope. But we would want to
successfully commit it by hand if we've succeeded. We could do that by adding
the following linear Drop</code> implementation:</p>
impl </span>linear Drop for Transaction {
</span>    </span>fn </span>drop</span>(linear </span>self</span>) {
</span>        </span>self</span>.</span>abort</span>(</span>self</span>);
</span>    }
</span>}
</span></code></pre>
This would make it so if the transaction goes out of scope or if we call abort</code>
it aborts. But only if we call commit</code> manually does it complete successfully:</p>
fn </span>do_thing</span>() -> Result<()> {      </span>// 1. No linear arguments, so no `linear fn` needed.
</span>    </span>let</span> txn = Transaction::new();  </span>// 2. Create a new transaction.
</span>    </span>some_action</span>()?;                </span>// 3. On error, drop `txn` and call the destructor.
</span>    txn.</span>commit</span>();                  </span>// 4. On success, call `txn.commit`.
</span>    Ok(())                         </span>// 5. Return from the function; no linear types are live.
</span>}
</span></code></pre>
Admittedly I'm not entirely sure whether linear Drop</code> in fact requires taking
linear Self</code> to be destructed, or whether it would suffice for Drop to keep
taking &mut self</code> as an argument, assuming that if the destructor is called, it
will be destructed anyway. The most important difference between linear Drop</code>
and regular Drop is that linear Drop will not run if the type is manually
destructed. And that unlike regular Drop, linear types guarantee</em> they will be
destructed. The exact details of the interfaces are secondary to the semantics
we're trying to encode.</p>
Control</h2>
In this section I'll be covering the interactions of linear types with
control-flow primitives such as panic</code> and async</code>. Creating a coherent model
for their interactions is a requirement to make linear types practical.</p>
Control Points</h3>
You may have heard of the term "control flow" to refer to things like branches,
loops, and function calls. It describes the "flow" of control over execution.
Perhaps you've also heard of "control flow operators" to refer to things like
if</code>, while</code>, and for..in</code> which are the specific keywords we use to encode
control flow operations. In the compiler all of these things are tracked in a
data structure called the "control-flow graph" (CFG).</p>
I've found there is a term here missing, which is something I've started
calling "control points": places in the code where the callee hands control over
execution back to its caller. The caller then may or may not hand control back
to the callee 8</a></sup>. Not all control-flow operations are equal
though. We can divide them into two categories:</p>
^{8</sup>
This does not include points at which control over execution
is handed off to another function by calling it. In structured programming
languages such as Rust we can treat those as black boxes, so from the function's
perspective we only have to observe what happens when they hand control back to
us again.</p>
</div>}
Breaking operations</strong>: hand control back to the caller, and that's it. Examples:
return</code>, break</code>, continue</code>, ?</code>, panic!</code>.</li>
Suspending operations</strong>: hand control back to the caller, but the caller can
choose to hand control back</em> to the callee again later. Examples:
.await</code>, yield</code>.</li>
</ul>
I'm using "caller" and "callee" somewhat loosely here, since Rust is
expression-oriented and with things like try {}</code> blocks and inline loop {}</code>,
control flow doesn't necessarily need to be handed to a different function. It
can be enough to be handed to an enclosing scope. Let's take a look at a
reasonably representative Rust function which does some async IO and then some
parsing:</p>
async </span>fn </span>do_something</span>(</span>path</span>: PathBuf) -> io::Result<Table> {
</span>    </span>let</span> file = fs::open(&path).await?;
</span>    </span>let</span> table = </span>parse_table</span>(file)?;
</span>    Ok(table)
</span>}
</span></code></pre>
And here it is again, with all control points fully annotated:</p>
async </span>fn </span>do_something</span>(</span>path</span>: PathBuf) -> io::Result<Table> {
</span>                                        </span>// 1. futures start suspended, and may not resume again
</span>    </span>let</span> file = fs::open(&path).await?;  </span>// 2. `.await` suspends, and may not resume again
</span>                                        </span>// 3. `?` propagates errors to the caller
</span>                                        </span>// 4. `fs::open` may panic and unwind
</span>    </span>let</span> table = </span>parse_table</span>(file)?;     </span>// 5. `?` propagates errors to the caller
</span>                                        </span>// 6. `parse_table` may panic and unwind
</span>    Ok(table)                           </span>// 7. return a value to the caller
</span>}
</span></code></pre>
While many of the control points in this function are visible, many others are
not. In order for linear types to function, they need to account for all</em> control
points regardless of whether they're visible or not. This is why it's so
important to be able to implement linear Drop</code> destructors. Because without
them it wouldn't be possible to carry linear types over control points.</p>
Liveness</h3>
Now that we've covered what control points are, we can discuss how they interact
with linear types. In its most basic form: when a linear type is held live
across</em> a control point, the compiler cannot prove that it will be consumed.
Assuming we take our earliest version of Transaction</code> which didn't implement
linear Drop</code>.  If you hold a an instance of it across an .await</code> the compiler
should throw an error. The ?</code> may cause the function to return, meaning there
is no guarantee txn.commit</code> will ever be invoked:</p>
fn </span>do_something</span>() {
</span>    </span>let</span> txn = Transaction::new();  </span>// 1. Create an instance of `Transaction` (assuming no `linear Drop`).
</span>    </span>some_action</span>()?;                </span>// 2. ❌ `?` may cause the function to return with an error.
</span>    txn.</span>commit</span>();                  </span>// 3. ❌ we cannot guarantee `txn.commit` will be called.
</span>}
</span></code></pre>
This is the reason why in order for linear types to be practical, they must</em>
implement a form of "consuming Drop". Without it we couldn't hold any linear
types across control points. And because every function in Rust is currently
allowed to panic, it means having linear types without destructors would be
entirely impractical</strong>. That's why its likely virtually every linear type will
want to implement some form of linear Drop</code>, and there are open questions on
how deeply we could integrate this into the language:</p>
fn </span>do_something</span>() {
</span>    </span>let</span> txn = Transaction::new();  </span>// 1. Create an instance of `Transaction` (assuming `linear Drop`).
</span>    </span>some_action</span>()?;                </span>// 2. ✅ `?` may cause the function to return with an error, invoking `txn.drop`.
</span>    txn.</span>commit</span>();                  </span>// 3. ✅ Or `?` does not return, and `txn.commit` is called.
</span>}
</span></code></pre>
The key interaction between linear types and control points is one of
liveness</em>. When interacting with a control point, linear types must guarantee
they will remain live</em>. This means the linear type must either being returned
from a scope via the control point, or must be able to be destructed before the
control point finishes handing control back over (e.g. linear Drop</code>). I hope
the terminology of "liveness" and "control points" can provide us with the
missing vocabulary to talk about how linear types interact with things such as
panics and errors.</p>
Abort points</h3>
It's important to distinguish between "control points" and what I'm calling
here: "abort points". Where control points represent points in the control-flow
graph where control over execution is handed off to another place within the
program. "Abort points" represent points within the program at which point the
program stops executing entirely.</p>
In practice, every line of code represents an abort point. There is no guarantee
a computer will execute the next instruction on a CPU. The best it can do is
guarantee that if</em> a next instruction will be run, which one it will be. Power
plugs can be pulled, CPUs can shut down processes, kernels can crash. These are
things which live outside the type system, and we cannot model within the type
system. The type system assumes that the hardware it runs on is well-behaved. If
it isn't, the model doesn't hold.</p>
When we write panic=abort</code> in our Cargo.toml</code> it changes the meaning of
panic!</code> from "unwind" to "just immediately stop". Similarly: we can use
std::process::exit</code></a> to
stop a program then and there. Because we exit the confines of the program, the
type system cannot meaningfully say anything about it, so linear types can't
model it.</p>
Similarly: we can't treat slow</em> operations as control points either. If a
program runs thread::sleep(10_years)</code>, as far as the type system is concerned
that doesn't represent a control point anymore than thread::sleep(10_nanos)</code>
would. The only time we can really capture that a linear type is violated is in
the case of unreachable</em> code. Take for example the following example:</p>
fn </span>do_something</span>() {
</span>    </span>let</span> txn = Transaction::new();  </span>// 1. Create an instance of `Transaction`.
</span>    </span>loop </span>{}                        </span>// 2. Loop infinitely
</span>    txn.</span>commit</span>();                  </span>// 3. Complete the transaction.
</span>}
</span></code></pre>
… which should probably produce an error like this:</p>
error: linear type destructor is unreachable
</span> --> src/lib.rs:4:1
</span>  |
</span>3 | loop {}
</span>  | ------- any code following this expression is unreachable
</span>4 | txn.commit();
</span>  | ^^^^^^^^^^^^^ unreachable destructor
</span>  |
</span></code></pre>
Async and Linearity</h2>
Async Rust provides us with the ability to write non-blocking code in a
imperative fashion, and through its design unlocks two key capabilities:</p>

The ability to execute any number of async operations concurrently</em>.</li>
The ability to cancel</em> any operation before it completes by dropping its
corresponding future.</li>
</ol>
"async", much like "linearity" can best be thought of as an effect</em>. And when
we talk about combining async an linearity, what we're really discussing is
effect composition</em>. Effects are built into the language, and hold a deep
semantic relationship to the underlying program. This means that when they
interact we get to choose how they should behave together. In the case of
composing linear and async, there are two behaviors we want to be able to
encode:</p>

must-await futures</strong>: these are futures which must be awaited, but once
awaited may still be cancelled. Because cancellation is a very useful
property for most kinds of futures.</li>
must-consume futures</strong>: these are futures which not only must be awaited,
even if attempted to be cancelled, they must continue doing their work.</li>
</ol>
In order for a future to be .await</code>ed</a>, it must first be
"stack-pinned"</a>, resulting in a Pin<&mut T></code> type. In the
general case this means we've permanently lost access to the inner T</code> type:
once a type has been pinned, it must remain pinned until it's dropped. This
means that once we start polling a future, we can no longer destruct it.
That's why for async Rust we need to expand the destructor rules of linearity
slightly. In order to "consume" a linear future, it should be enough to place it
inside a .await</code> expression. Awaiting a future should be considered equivalent
to consuming a future, even if we cancel it later on. This provides us with "must-use"
futures:</p>
let </span>_ = </span>meow</span>();  </span>// ❌ "`meow` returns a `Future` which must be `.await`ed"
</span></code></pre>
In order to encode "must-consume" futures we will need access to an
async version of linear drop. As Sabrina Jewson</a> pointed out before: if we
require a future to continue doing work when cancelled, it can continue doing
that work in its drop handler. If the drop handler itself is un-cancellable,
then we are able to construct un-cancellable futures.</p>
Unlike the linear Drop</code> we've shown before, async linear Drop</code> cannot take
self</code> by-value because of the pinning rules. It instead it can only really take
&mut self</code>. So again, the fact that it's being destructed will need to be
guaranteed by the fact that it's .await</code>ed. Encoding the linearity of async linear Drop</code> itself is a different matter. And I suspect we'll need to dedicate
a separate post to exactly spell out how that should work. My guess is that this
will require a connection that is as deep to the language as non-async Drop</code>.
And that at the root of the program it will bottom out in something like it
being a linearity violation if you don't handle the future returned by Drop</code> to
completion. I'm not exactly sure yet how this should be done, but it seems like
it should be doable.</p>
Cycles and Safety</h2>
Unsafe Rust provides access to raw pointers. These are pointers, which unlike
references, may alias if you're not careful. But just because they could</em> alias
doesn't mean they're allowed to alias. And it's in fact undefined behavior if you
fail to uphold Rust's safety rules in unsafe Rust. unsafe</code> only removes the
guard rails, it doesn't change the rules.</p>
The same would apply to linear fn</code>/linear T</code> and unsafe</code>. It should be
possible to cast a linear T</code> to a regular T</code>, if you make sure to keep
treating T</code> as if it was a linear T</code>. This would allow you to create linear
functions which internally could hold cycles, but which externally would not be
observable. Assuming we had a "make this type non-linear" (UnsafeNonLinear</code>)
equivalent to
UnsafeCell</code></a>. We
could imagine we could take an instance of Transaction</code> and place it inside an
Arc</code> somewhat like this:</p>
let</span> txn = Transaction::new();                     </span>// create a new linear transaction
</span>let</span> txn = </span>unsafe </span>{ UnsafeNonLinear::new(txn) };   </span>// cast it to remove its linearity
</span>let</span> txn = Arc::new(txn);                          </span>// place it inside an `Arc`, *may* lead to cycles
</span>let</span> txn = Arc::into_inner().</span>unwrap</span>();             </span>// extract it from the `Arc`
</span>let</span> txn = </span>unsafe </span>{ txn.</span>into_inner</span>() };            </span>// convert it back to a linear type
</span></code></pre>
I don't really know whether it makes most sense to create an unsafe wrapper
type, or whether we could use as</code>-casts to peel the linearity effect. I would
expect casting a T</code> -> linear T</code> would probably be safe, but I don't know for
sure. Because this would be a new language construct we would be able to define
all of these rules. They seem like they should be fairly straightforward, so
fingers crossed it actually ends up being that way.</p>
Finally there's an open question we have is whether it could in fact be possible
to create linear versions of Arc</code> and Rc</code> (and Mutex</code>/RefCell</code>). If every
cloned instance had to prove it was consumed, would it be possible to guarantee
that no cycles may occur?  Even if that's not possible for the existing APIs,
perhaps a different formulation could provide those guarantees. Something like a
scope</code> API might be able to provide such guarantees. If this were possible it
would make linear APIs significantly more convenient to use, so I suspect it
would be worth examining this further.</p>
Closure captures</h2>
An interesting challenge Gankra shared on Twitter</a> is how
closure captures interact with linearity. In order for "linear Rust" to be as
ergonomic as non-linear Rust, the two should feel largely similar when used.
This means closures and closure captures should be accessible from linear Rust
too.</p>
Let's examine an example more closely. Here is the Option::map</code> function as
implemented in the standard library today. It takes an Option</code>, and a closure, and
if Some</code> applies the closure to it. If we take the None</code> branch, at first
glance it seems like nothing will happen. But on closer inspection we may
realize that we drop the closure, and so we also drop all variables captured by
the closure:</p>
impl</span><T> Option<T> {
</span>    </span>pub fn </span>map</span><U>(</span>self</span>, </span>op</span>: impl FnOnce(</span>T</span>) -> U) -> Option<U> {
</span>        </span>match </span>self </span>{
</span>            Some(t) => Some(</span>op</span>(t)),
</span>            None => None,
</span>        }
</span>    }
</span>}
</span></code></pre>
In Niko's blog post, this example was adapted to use T: ?Drop</code>. This is valid
because in the Some</code> case the T</code> parameter is passed to the closure, which can
consume the linear type. While in the None</code> case there is no T</code>, so there is
no type to consume. But as we've mentioned this does not account for the
variables captured by the closure, which in this formulation would likely just
be disallowed from performing any captures. Instead, using the linear</code> effect
we can model this the way we would expect because we have access to linear
destructors, and because a linear type can only be captured by another linear
type:</p>
impl</span><linear T> Option<T> {
</span>    </span>pub</span> linear </span>fn </span>map</span><U>(</span>self</span>, </span>op</span>: impl linear FnOnce(</span>T</span>) -> U) -> Option<U> {
</span>        </span>match </span>self </span>{
</span>            Some(t) => Some(</span>op</span>(t)),
</span>            None => None,
</span>        }
</span>    }
</span>}
</span></code></pre>
Because T</code> is linear, the closure FnOnce</code> which takes T</code> as an argument must
also be linear, and in turn the map</code> method must be linear as well. This
does assume that linear closures can be dropped to run the destructors of the
variables they've captured. Which seems reasonable, since the Fn</code>-family of
traits are marked #[fundamental]</code> and are part of the language - so we can
define how they should function in the presence of linear</code>.</p>
Linear stdlib</h2>
Our linear fn map</code> example doesn't quite capture the same semantics as Niko's
proposal yet. This syntax requires</em> types to be linear, whereas in Niko's
version types may</em> be linear - or they may not be. In order to capture this
"maybe-ness" of effect keywords we need a way to express it: which is what the
Keyword Generics Initiative has been working
on</a>.</p>
To make our effect-based design of Option::map</code> fully compatible with the
semantics Niko laid out, we'd have to make one more change. Rather than using
T: ?Drop</code> which communicates the presence of an effect through the absence of an
auto-trait, we can instead use ?linear T</code> to more directly communicate the
potential presence of an effect:</p>
impl</span><</span>?</span>linear T> Option<T> {
</span>    </span>pub </span>?linear </span>fn </span>map</span><U>(</span>self</span>, </span>op</span>: impl ?linear FnOnce(</span>T</span>) -> U) -> Option<U> {
</span>        </span>match </span>self </span>{
</span>            Some(t) => Some(</span>op</span>(t)),
</span>            None => None,
</span>        }
</span>    }
</span>}
</span></code></pre>
From the feedback we've received I know many people are not particularly
thrilled about this syntax - which is perfectly understandable. But exact syntax
aside, it's important to observe that if we do this right, then this syntax
should be mostly temporary. Aside from a few exceptions in the stdlib (Arc</code>,
Rc</code>, mem::forget</code>), most library code can probably be marked as linear</code>.
Which begs the question: if most code can be linear, could it make sense to
eventually assume linearity, unless people opt-out? If that were the case, then
the signature of Option::map</code> would remain identical to what it is today in the
stdlib already. But the signature of mem::forget</code> could then become:</p>
pub const fn </span>forget</span><T: </span>!</span>linear>(t: T) { .. }
</span></code></pre>
It seems far more desireable to gradually linear</code>-ify the stdlib rather than
defining a parallel linear</code> stdlib checked into crates.io</code> (as we've done once
already</a> for async</code>). And far more
empathic than trying to argue that users of Rust are in fact wrong for wanting to
make use of APIs such as Option::map</code> in linear contexts. Regardless of which
set of effects you're using in Rust to program with, familiar patterns and
techniques should be accessible to all.</p>
Movability</h2>
In his post Niko describes "linear types" as "must-move types". This begs the
question: if there are types which must</em> move, are other kinds of moves
possible as well? And there are. By default types in Rust can be described as
"may-move types", because they can be moved but they don't have to. And then
we also have access to "must-not-move types" which are represented by combining
the Pin</code></a> type with the
!Unpin</code></a> trait. Though
in practice most people prefer to encode the immovability of types
using the more declarative pin-project</code>
crate</a> instead.</p>

must move</strong>: linear types</li>
may move</strong>: the default mode of Rust types</li>
must not move</strong>: pinned types</li>
</ul>
Thinking of "must-move types" (linear) as the opposite of "must-not-move types"
(pinned) is a useful model to carry forward, as they encode opposing guarantees
about the movability of types. Reconciling these opposing guarantees is what
makes combining async + linear types difficult, since once futures start being
polled can generally no longer be moved. If this post continues into a series,
this would be a topic I'd like to revisit in more detail later on.</p>
Further Reading</h2>
Gankra wrote an excellent post in 2015 on why mem::forget</code> should be
safe</a>, explaining why we
can't assume destructors will always run in Rust. She wrote a follow-up post in
2017 about what makes linear types hard to
model</a>, concluding on an optimistic
note with: "Ok having actually written this all out, it’s better than I
thought."</p>
The Wikipedia page on linear
types</a>
provides a useful overview of the difference between among other "linear" and
"affine" types. Including type systems with both more and less guarantees than
both. Phil Wadler's "linear types can change the
world!"</a>,
provides a great type-theoretical introduction to what linear types are, and why
they are useful. Similarly on type-theory side, Phil Wadler also published
"Propositions as Types"
(paper</a>,
talk</a>), making an excellent case
that properties such as lambda calculus and linear types are not invented - they
are discovered. Meaning they may in fact be a fundamental part of logic and
computing.</p>
More recently Tyler Mandry wrote a blog post on the challenges of defining
scoped tasks in
Rust</a>. And Niko
Matsakis wrote a post on "must move"
types</a>,
as a potential way to overcome the challenges scoped tasks present. Miguel Young
de la Sota gave a talk about move constructors in
Rust</a>, explaining how like in C++,
Rust could implement self-referential types which can be moved. Yosh Wuyts (hello)
wrote a
post</a> on
the challenges we face in order to make common interactions with pinned values
safe. Sabrina Jewson wrote a post describing the challenges of async
destructors</a>,
going into great detail about various considerations in the design space.  And
Yosh Wuyts (hello, again) wrote a post describing how async
cancellation</a> works, and why
it's an essential capability of async Rust.</p>
Conclusion</h2>
Linearity in Rust is largely inexpressible today. This leads us to work around
this restriction in various ways, which come at the cost of correctness,
performance, and ergonomics. In this post we've shown how linearity in Rust
could be modeled as an effect, bypassing any of these restrictions. We've shown
how it could be used to create ergonomics which are largely similar to existing
Rust, and how we could gradually introduce linearity into the stdlib and
ecosystem.</p>
We've shown how for the sake of ergonomics it's not only important to be able to
create and consume bare types, but also to be able to define constructors and
destructors which must be called as pairs. Another contribution this post has
made is formulated a model for linear Drop</code>; which is required to preserve
Rust's existing ergonomics in a linear context.</p>
Finally we've taken a closer look at how linearity interacts with cycles, with
unsafe</code>, and with async</code>. For async</code> specifically, we've taken a look at how
linear async fn</code> would work, why it should not disallow cancellation by default,
and how we could use it to still encode "must-not cancel" semantics.</p>
To close this post out: I don't expect this post to function as a definitive design.
In fact, it was written with the opposite intent. I hope this post will help us
better understand the challenges linear types face in Rust, and serve as a
useful tool in getting us closer to a design which we would be happy to adopt.</p>
I'd like to thank Sy Brand, Iryna Shestak, and Eric Holk for reviewing this
post prior to publishing.</em></p>

Using HTML as a compile target
Fri, 03 Feb 2023 00:00:00 +0000
If you've been around an industry for long enough you end up with Opinions™
about things. Here's one of mine: I think one of the biggest missing aspects of
web programming is the lack of support for treating HTML as a compile-target.</p>
Most folks reading this blog may not be aware of this, but I started my
career working with web technologies, which I kept at for the better part of a decade.
I co-authored one of the (at least to my knowledge) first 1</a></sup> CSS-in-JS engines</a>,
I've written frontend frameworks</a>, designed web
compilers</a>,
and authored entire HTTP stacks</a>. Even if
web tech isn't really my current area of focus, it's something I've worked on a
fair bit, and still care about a lot.  So here's a post about an opinion I have,
in a field I don't do much in currently, but still think would like to share.</p>
^1</sup>To my knowledge at the time at least. I'll totally buy that e.g. PHP
or Google's web compiler may have supported similar stuff - but we didn't know
about it at the time. When we designed sheetify in ~2015 ish css-modules</code> was
all the rage, and we figured we could do something similar without needing to
overload the meaning of require</code>. And we didn't know of anything that worked
similar at the time. Later,
styled-components</code></a> became a thing, citing our
work on sheetify as an inspiration. But that's all ancient history at this
point.</p>
</div>
A taste of HTML</h2>
There are well over 100 unique HTML elements in the spec. Let's zoom in on just
one of those: the <li></code> element</a>.
It inherits from a class hierarchy which is 4 levels
deep</a>. It
exposes two unique attribute values, of which 1 has been deprecated and
shouldn't be used. And it can't just be placed anywhere in the DOM: it has
exactly 4 valid parent elements, of which 1 also has been obsoleted. Here are
some examples of it in action:</p>
<</span>aside</span>>
</span>    <</span>li</span>>first</</span>li</span>>
</span>    <</span>li</span>>second</</span>li</span>>
</span></</span>aside</span>>
</span>
</span><</span>menu</span>>
</span>    <</span>li</span>>first</</span>li</span>>
</span>    <</span>li</span>>second</</span>li</span>>
</span></</span>menu</span>>
</span></code></pre>
Quiz time: which of these is valid? Which of these isn't? None, both, either
one? When you write the code, you don't know. It's only when you try and render
the code in the browser that you'll receive feedback.</p>
Okay fine, the answer to the last question was that <aside><li></code> is invalid,
but <menu><li></code> is valid. Quiz number two (no peeking at the reference): what
is rendered here?</p>
<</span>ol</span>>
</span>    <</span>li </span>value</span>="</span>2</span>">second</</span>li</span>>
</span>    <</span>li </span>value</span>="</span>1</span>">first</</span>li</span>>
</span></</span>ol</span>>
</span>
</span><</span>menu</span>>
</span>    <</span>li </span>value</span>="</span>2</span>">second</</span>li</span>>
</span>    <</span>li </span>value</span>="</span>1</span>">first</</span>li</span>>
</span></</span>menu</span>>
</span></code></pre>
Okay, answer time: first one is valid, second one is not. But both will render,
without error. It's just that in the first case value</code> is respected ("first,
second") while in the second case it renders "second, first". Did you know that?
I sure didn't.</p>
As you're probably aware I'm not actually trying to teach you the rules of
<li></code>, but just trying to show that this teeny tiny basic element has a host of
interactions that will affect what you build. And we're only scratching the
surface of this element. It also interacts with styling, event handlers, ARIA roles,
and elements contained within. The fact that value</code> attributes on <li></code> are
valid when nested within <ol></code>, but not within <menu></code> is just one interaction
of thousands out there. Everyone gets things wrong all the time with this. And
we need to do better. All these rules are specced, implemented, and (mostly)
consistent. We need a way to automate this knowledge.</p>
Type-Safe HTML</h2>
I think we need to start by acknowledging that HTML is complex. Not in a
deregatory way, HTML is also really useful, but as a statement of fact. There
are more rules and interactions than anyone reasonable person can manage. Which
means we shouldn't expect people to be the ones managing it.</p>
Instead, I believe that HTML would benefit from type safety</em>. Right now the
way HTML is validated is by shipping it to a browser, and then checking the
console tab for errors. Or sometimes people run linters on it such as
lighthouse</a> or pa11y</a>. Sometimes these things even happen in CI.</p>
A shortcoming of this approach is that in order to validate whether you've done
things right, you first have to load your web page into a browser engine, and
only then you can validate whatever was rendered. But that's where it stops: it
only validates what was rendered at that point. If your page has different
components which can be in different states, or also has different pages, you
need to ensure every one of those is validated. And that's often more work than
it's worth, so people don't bother.</p>
Instead the way I'm thinking about it is: what if we moved the validation to a
compilation pass. That way we can prevent errors from occurring altogether. A
compiler is able to reason about all states, and validate they're all valid, all
in a blink of an eye. No browsers required. For people who're looking to reduce
the amount of runtime issues in their websites: this is by far the most feasible
way to actually get the number of HTML bugs down to zero.</p>
This isn't the first time I'm writing about this: I first
pitched this in 2019</a> because I didn't have the bandwidth to implement it
myself. But Bodil did, who ran with it and subsequently published the
typed-html</code> crate</a>. This is good; I'd love to see different approaches of this
same idea spring up in Rust and in other languages. All that's needed to get started
is an idea of what the resulting API should look like. And then write a compiler
which takes .webidl</code> node definitions</a>, and outputs types in the right
shape on the other end 2</a></sup>.</p>
^{2</sup>
It'd be amazing if the W3C kept a repo with these definitions up to
date themselves, so everyone else could just pull them in. But one can wish.</p>
</div>}Type-Safe Semantic Represenation</h2>
Once you have a base of checked HTML types that's pretty good. But that doesn't
mean we're quite done yet. Remember when I said we weren't getting into ARIA
roles? Well, we are now.</p>
So see, if HTML is structure</em>, and CSS is representation</em>, then ARIA is
"semantics". WAI-ARIA</a>, or "ARIA roles" for short, enables you to ascribe meaning to
your structure. When you see a bunch of <li></code> items in text form, what are you looking at?
Are they tabs? Are they a menu? Are they a dropdown? ARIA roles enable you to
meaningfully distinguish between them. How many of these roles are there? Over
250, and they all have different properties and interactions.</p>
Again: the path away from trouble here is by creating abstractions. Instead of having
to remember which elements to apply the tablist</code></a>, tab</code></a>, and tabpanel</code></a>
roles on, you should be able to create something akin to Swift's TabView</code></a>
which encapsulates all of those differences for you.</p>
This is a higher level of abstraction than just typed HTML provides. But it's
equally important, since it's a requirement for useful keyboard navigation and
accessibility. It'll take some more work than just plain typed HTML, and some
more design work is required to incorparate best practices on how to actually
structure elements since you can't just scrape a spec for definitions. But once
you can reason about e.g. "tabs" or "feeds", building more complex things
becomes a lot easier. Check out the ARIA Patterns
page</a> to get a sense of the kind of
stuff this could cover.</p>
Embracing Compilation</h2>
To recap so far: we've established that HTML and ARIA are complex</em> interfaces, that
checking during compilation rather than at runtime has a lot of benefits, and
that applying this to ARIA it would enable us to reason about elements at a
higher level of abstraction.</p>
I believe that in order to make this all make the most sense, the web is missing
"sourcemaps for HTML". If we're going to be compiling a bunch of semantic
components down to HTML, then devtools should enable us to reason about them as
components</em>. When inspecting HTML in the DOM, we shouldn't have to look at a
bunch of <div></code>, <p></code>, and <span></code> tags. But more meaningful things, such as
<feed></code>, <profile></code>, and <calendar></code>.</p>
The way I think this is that devtools are like having access to a debugger. When
debugging a compiled program we usually don't reason in terms of MOV rax jax</code> and
POWF bonk donk</code>. But the operations we actually wrote such as: pet_cat</code>, and
big_stretch</code>. It's relatively infrequently that actual output is what we're
interested in, but it's always there if you need it.</p>
Some of you web heads might be wondering: "What about Web Components? - Isn't
that the same?"</em> And, like, kind of! It's similar in how a reflection in a
mirror is similar to the thing it reflects: similar at first sight, but actually
everything is the opposite.</p>
The point of the Custom Elements part of Web Components is that you can send
<custom-tags></code> to the browser which are then interpreted at runtime to
introduce interactivity, but preserve qualities such as cacheability of shared
resources. A canonical example here is the relative-time-element</code></a> tag used by
GitHub. What we're doing is the opposite: the HTML we send to the browser is
already fully expanded, and doesn't need to interpret any JS to enable interactivity.
What we want is for the browser to be able to interpret all of our <li></code> and
<span></code> tags generated by our components as the <calendar></code> and <feed></code>
elements that we've authored. This is much closer to source maps: associate
the building blocks we generated with the semantic components they actually came from.</p>
The similarity with custom elements comes back up when we take a look at
how custom-elements are rendered in the browser's Elements tab:</p>
<</span>relative-time
</span>    </span>tense</span>="</span>past</span>"
</span>    </span>datetime</span>="</span>2016-11-23T11:12:43-08:00</span>"
</span>    </span>data-view-component</span>="</span>true</span>"
</span>    </span>title</span>="</span>Nov 23, 2016, 8:12 PM GMT+1</span>">
</span>    #shadow-root (open)
</span>        "6 years ago"
</span>    November 23, 2016 11:12
</span></</span>relative-time</span>>
</span></code></pre>
This contains all the information anyone needs to debug the element. And what
I'd love is to be able to access this view for server-rendered components
without ever needing to go use any JS. And the uses don't stop there either:
frameworks such as React also reason in terms of components and require custom
devtools</a> to re-surface that view. What we really need is a way
to ascribe semantic meaning to our elements in a way that works for both server-
and client-side applications.</p>
Now, it could be that I've missed a capability that would enable doing
exactly</em> this with custom elements in browsers today - or perhaps some other
mechanism. It's been a minute since I last used custom elements. But if that's
the case I'd love to learn about it, because in my opinion every framework under
the sun should start making use of that. I know I will!</p>
Conclusion</h2>
In this post we've talked about the complexity of getting HTML right, how using
a compiler could be used to mitigate those issues, what ARIA roles are, how we
could model higher-level type-safe components, and finally what's missing in the
browsers devtools to round this out.</p>
I strongly believe that type-safety would be a boon for the browser space.
People have already experienced some of it with the adoption of TypeScript, and
I believe leaning further into types and type-safety would enable the web
platform to become far more accessible - both for developers and users.</p>
The only piece we can't directly control is the ability to teach browsers to
reason about the semantic meaning of elements - almost like being able to
declare source maps for the resulting HTML. In an alternate universe this is
what Custom Elements</a> would have been. But I believe in order to make the web
platform more accessible, we must make it possible to actually reason about
<feed></code> and <calendar></code> as first-class items. And not just the <div></code> and
<p></code> tags they're comprised of.</p>
update (2023-04-07):</strong> As coincidence will have it, two weeks after publishing
this post the chrome</a> and
safari</a> teams
began discussing the upcoming "declarative shadow DOM" feature.
This seems like it will resolve the outstanding issues involving shipping semantically meaningful elements
from the server.</p>
update (2023-04-11):</strong> I published the html</code> crate</a>
implementing most of this blog post. You can find the source on my GitHub at
yoshuawuyts/html</a>.</p>

Capability-Safety I: Prelude
Tue, 24 Jan 2023 00:00:00 +0000
Introduction</h2>
In 2021 Tmandry published the post: "Contexts and Capabilities"</a>.
In it they present the idea of with</code>-clauses, providing a means of passing types such as allocators around in a more
convenient way. The post goes into detail about the feature, but you can
roughly think of it as a way to declare types which are live within a certain
scope, and are automatically passed into functions which require them:</p>
//! A usage-only example of `with`-clauses to create scoped allocators.
</span>//! This example is somewhat incomplete, but hopefully gives enough of a sense
</span>//! of what it would feel like to use `with`-clauses.
</span>
</span>// Before: using `#[feature(allocator_api)]` to allocate into a bump allocator.
</span>let</span> bump = bumpalo::Bump::new(); </span>// Create a bump allocator
</span>let</span> vec1 = Vec::new_in(&bump);   </span>// Use `new_in` to allocate into the bump allocator
</span>let</span> vec2 = Vec::new_in(&bump);
</span>
</span>// After: using `with`-clauses to allocate into a bump allocator.
</span>with std::alloc::alloc = bumpalo::Bump::new() { </span>// Create a bump allocator for this scope
</span>    </span>let</span> vec1 = Vec::new(); </span>// `new` uses whichever allocator is currently active
</span>    </span>let</span> vec2 = Vec::new();
</span>}
</span></code></pre>
Capability-Safety</h2>
While we were brainstorming about with</code>-clauses it got me wondering: Would it be possible to not
only leverage with</code>-clauses as a means of just passing arguments around, but
also as a means of guaranteeing encapsulation</em>?. Or differently put: if we can
think of with</code>-clauses as a way of passing capabilities, would it be possible
for us to make Rust "capability-safe" 1</a></sup>? I reached out to
Sunfishcode</a> who has worked a lot on capabilities for WebAssembly, and for
the past year or so we've been researching how we might be able to introduce
a notion of capability-safety into Rust.</p>
^{1</sup>
I'm intentionally not really going into detail about what
"capability-safety" is exactly; a follow-up to this post is being written. But
to not leave people entirely in the dark, let me
try and summarize it: What if we had access to a kind of function which
guarantees it will only ever operate over exactly</em> what it declared in its
function signature and nothing else. This enables functions to not only
describe what they can</em> do, but by omission now also what they can't</em> do.
A lot of safety-critical systems would love to have access to that kind
of expressiveness, but there are a bunch of other uses for it too. This
intersects with a bunch of different aspects of the language, so expect us to go
into minute detail about how we believe we can make this all work in future posts.</p>
</div>
The good news is: we think we have a plausible design - one which not only
enable capability-safety, but crucially would so without breaking any of Rust's
stability guarantees. We've consulted with members of both lang and libs, and
at least from a technical point of view it seems like it could work. But if we ever want to
turn it into a formal Rust language proposal, we not only need
the design to be plausible</em>, we also need to show that it's something which is beneficial</em>.
Something which solves real issues people experience, and something which people
would be excited to adopt.</p>}The plan</h2>
You can consider this post a statement of intent. Sunfish and I are committed to
figuring out how we can actually talk about capability-safety in a way that
makes sense. I've tried and failed to write something coherent post about this a
couple of times now. But we believe it's time to actually start publishing about
this. So that's exactly what we'll be doing!</p>
We'll start explaining by what capabilities are, why they're
useful, what capability-safety is, which guarantees it provides, introduce you
to our design, and then work through the various interactions. There's not a lot of
material on capability-safety available, so the real challenge for us is to
figure out the right framing and examples to have it actually make sense.</p>
But we're convinced we can actually make it happen. Sunfish has already started
writing about some these topics which you can check out here:</p>

No ghosts!</a> (2022-03-15)</li>
What is a capability?</a> (2022-11-21)</li>
</ul>
His next post will be about capability-safety. Stay tuned for that!</p>
Thanks to Sunfishcode</a> for help reviewing this post!</em></p>

domain-specific error macros
Tue, 17 Jan 2023 00:00:00 +0000
When I write custom errors in a project, I also like to write a few small error
macros to accompany them. In my opinion this can make error handling just a
little nicer to use. In this post I briefly want to talk about domain-specific
error macros such as ensure!</code>, what they're useful for, and the io-ensure</code></a>
prototype crate I've written which I'm propose for inclusion in the stdlib next
time I get the chance.</p>
What is ensure?</h2>
If you use Rust you're probably familiar with
assert!</code></a>: it checks a
condition, and if the condition doesn't match it panics. This often looks
something like this:</p>
let</span> condition = </span>1 </span>== </span>1</span>;
</span>assert!(condition, "</span>numbers should equal themselves</span>");
</span></code></pre>
This works well if you're writing tests, or validating invariants - but when
you're validating input at runtime, instead of panicking you usually probably
want to return errors instead. And the error equivalent to assert!</code> is
ensure!</code>:</p>
fn </span>validate_buffet</span>(</span>s</span>: &</span>str</span>) -> io::Result<()> {
</span>    ensure!(s.</span>starts_with</span>("</span>tuna</span>"), "</span>the first course on the cat menu should be tuna</span>");
</span>    Ok(())
</span>}
</span></code></pre>
When discussing "ensure-style macros", these are typically the ones included:</p>

format_err!</code>: construct a new instance of an error using a format string</li>
bail!</code>: unconditionally create and return an error</li>
ensure!</code>: create and return an error if the condition doesn't match</li>
ensure_eq!</code>: create and return an error if two expressions don't match</li>
ensure_ne!</code>: create and return an error if two expressions match</li>
</ul>
bail</code> is kind of like panic</code>, and ensure</code> is kind of like assert</code>. You can't
really construct and pass around panics, so there is no panic-equivalent to
format_err</code>. Also for simplicity I'm omitting matches</code> variants, and errors
probably shouldn't be debug-only, so no debug</code> variants either.</p>
Pairing errors with macros</h2>
I believe it's good practice that if you're defining your own errors, to also
define your own error handling macros. Especially if it's intended to be
extended by third parties, it can make it significantly nicer to work with.</p>
The key here is that you can use ensure!</code> macros to encode domain-specific
logic with. Say you're writing an HTTP framework, you probably want all of your
errors to include an HTTP status code. While if we're dealing with IO, we
probably want to map back the underlying OS reason:</p>
// Example using `http_types::ensure!`
</span>ensure!(user.</span>may_access</span>(resource), StatusCode::</span>403</span>, "</span>user is not authorized to access {resource}</span>");
</span>
</span>// Example using `io-ensure`
</span>ensure_eq!(a, b, ErrorKind::Interrupted, "</span>we are testing the values {} and {} are equal</span>", a, b);
</span></code></pre>
new keywords</h2>
Work is happening in Rust on creating new language features to reason about
errors. areweyeetyet</a> provides an overview of this,
but the tldr is that we want to have a way to define "fallible functions" from
which you can "throw" errors. Also: no more Ok(())</code> at the end of every
function.</p>
Having access to a throw</code> (or yeet</code>) keyword means we'll have a canonical way
of returning an error from a function. ?</code> to re-throw errors, throw</code> to throw
new ones 1</a></sup>. But the bail</code> macro  does something very similar
already, and has the downside of having to be defined for every error type:</p>
^{1</sup>
I'm using "throw" in the Swift sense of the keyword: "checked
exceptions". Not the JS/Java/etc "unchecked exceptions" throwing; I think we all
like our Rust errors to be typed.</p>
</div>}// current
</span>return </span>Err(io::Error::new(e.</span>kind</span>(), format!("</span>{path}</span>: </span>{e}</span>")).</span>into</span>());
</span>
</span>// using `format_err!`
</span>return </span>Err(io::format_err!(e.</span>kind</span>(), "</span>{path}:{e}</span>").</span>into</span>());
</span>
</span>// using `bail!`
</span>io::bail!(e.</span>kind</span>(), "</span>{path}:{e}</span>"));
</span>
</span>// if we had `throw` in the language (feature composition!)
</span>throw io::format_err!(e.</span>kind</span>(), "</span>{path}:{e}</span>");
</span></code></pre>
What I'm thinking here is that it makes sense to include bail</code> if you're
writing your own libraries and applications. But probably less for the stdlib,
since whatever goes in there will need to remain stable forever, and having
bail</code> macros + throw</code> keywords is probably a bit too much.</p>
Stdlib</h2>
So I've written a quick prototype showing: "What if std::io</code> had domain-specific
macros" and published it to crates.io:
io-ensure</code></a>. Is it
great? I'm not sure. I think it's a decent quality of life improvement, and
perhaps it might make sense to include. Just like assert</code> can sometimes make it
easier to work with panics, ensure</code> might sometimes make it easier to work with
errors. Here's some random examples of IO error uses I translated from the
rust-lang/rust</code> repo:</p>

This example omits an extra inline call to format</code>, removing some of the nesting:</p>
// before
</span>fs::create_dir_all(parent).</span>map_err</span>(|</span>e</span>| {
</span>    io::Error::new(
</span>        e.</span>kind</span>(),
</span>        format!("</span>IO error creating MIR dump directory: </span>{parent:?}</span>; </span>{e}</span>"),
</span>    )
</span>})?;
</span>
</span>// with io macros
</span>fs::create_dir_all(parent).</span>map_err</span>(|</span>e</span>| {
</span>    io::format_err!(e.</span>kind</span>(), "</span>IO error creating MIR dump directory: {parent:?}; {e}</span>");
</span>})?;
</span></code></pre>
rust/compiler/rustc_middle/src/mir/pretty.rs</em></a> ←</p>

If librustdoc</code> also had its own error macros, we could imagine we could compose
it with IO error macros like so, saving us from some nesting and branching:</p>
// current
</span>if</span> nb_errors > </span>0 </span>{
</span>    Err(Error::new(io::Error::new(io::ErrorKind::Other, "</span>I/O error</span>"), ""))
</span>} </span>else </span>{
</span>    Ok(())
</span>}
</span>
</span>// with `librustdoc::ensure!` + `io::format_err!`
</span>ensure!(nb_errors > </span>0</span>, io::format_err!(io::ErrorKind::Other, "</span>I/O error</span>"));
</span>Ok(())
</span></code></pre>
rust/src/librustdoc/html/render/context.rs</em></a> ←</p>

This call to format</code> with the referencing and subslicing looks rather spicy.
I'm not super sure what's going on there, but presumably we should be able to
avoid it already? Either way, that's still an extra inline format</code> call gone:</p>
// current
</span>Err(io::Error::new(
</span>    io::ErrorKind::Uncategorized,
</span>    &format!("</span>failed to lookup address information: </span>{detail}</span>")[..],
</span>))
</span>
</span>// with `io::ensure!`
</span>Err(io::format_err!(
</span>    io::ErrorKind::Uncategorized,
</span>    "</span>failed to lookup address information: {detail}</span>"
</span>))
</span></code></pre>
rust/library/std/src/sys/unix/net.rs</em></a> ←</p>

Conclusion</h2>
In this post we've talked about domain-specific error macros, what they are,
what they're used for, and shown some examples of how they could be used for
io::Error</code> construction in the stdlib.</p>
Overall it's just a little convenience pattern which makes working with errors
nicer. Just like assert!</code> is a small quality of life improvement which makes
panics nicer to use; ensure!</code> is a small quality of life improvement which
can make errors nicer to use.</p>

Rust should own its debugger experience
Thu, 12 Jan 2023 00:00:00 +0000
Introduction</h2>
40% of Rust Developers</a> believe Rust's
debugging experience could use improvement. And that's not surprising: when we
write code we make assumptions, and sometimes we assume wrong. This leads to
bugs, which we then track down and fix. This is what we call "debugging", and is a
core part of programming. The purpose-built tools which help us with debugging
are called "debuggers".</p>
Unlike the Rust compiler, the Rust project doesn't actually provide a "Rust
debugger". Users of Rust are instead expected to use a third-party debugger such
as gdb</code>, lldb</code>, or windbg</code> to debug their programs. And support for Rust in
these debuggers is not always great</em>. Basic concepts such as "traits", "closures",
and "enums" may have limited support. And debugging async code, or arbitrary
user-defined data structures may be really hard if not impossible. This limits
the utility of debuggers, and in turn limits the Rust user's debugging
experience.</p>
Integration done right</h2>
When we interact with the "rust compiler", it's often through either cargo build</code>, or Rust-Analyzer in VSCode 1</a></sup>. The fact that a large chunk of
the compiler is in fact LLVM is for all intents and purposes just an
implementation detail. Even Rust-Analyzer powering the VS Code features is not
something you need to think about most of the time.</p>
This is how it should be. When you install Rust, the right version of LLVM gets
bundled in. The Rust project is responsible for LLVM doing the right things for
Rust, and we'll sometimes even float patches on a fork to fix things the project
depends on. Users should never become aware of these details. We're even at a
point where we might replace LLVM with a different backend in certain scenarios
2</a></sup>, and the only thing users should notice are faster build times.</p>
^1</sup>VS Code has 75% market share apparently. (src)</a></p>
</div>
^{2</sup>
I'm talking about Cranelift for debug builds here.</p>
</div>}Debuggers today</h2>
Debuggers have a very different story today. The Rust project doesn't really
provide a "debug workflow", meaning Rust users are made largely responsible for
figuring that out themselves. This means they need to find a debugger, install
it, and figure out how to run it. Whatever Rust support the debugger has is what
they get. And hopefully Rust is enough on debugger author's radars that support
for it improves over time. It's another question when newer versions of a
debugger may become available to Rust users too.  As a project we can contribute
some patches, but at a fundamental level we don't have any control</em> over these
tools. Which means we don't have any control over the resulting user experience.</p>
Work is happening though to enable the output of the Rust compiler to be better
understood by third-party debuggers. For example RFC 3191: Debugger
Visualizer</a> has been accepted which enables Rust users to author
debugger-specific visualizer scripts. This enables library authors to
define debugger-specific scripts which can provide more details about types.
This is a very pragmatic fix which will enable, say, the stdlib to immediately
start providing a better UX for existing debuggers. But we can't reasonably
expect all Rust projects to define debugger visualizations for all of their data
structures, so as a general solution it has some pretty clear limitations.</p>
Towards a coherent experience</h2>
In my opinion the Rust project needs to rethink its relationship to the debugger
user experience. Rather than something that we leave up to users to figure out,
it should be something that the Rust project is in charge of providing to Rust
users. If we want the Rust debugging experience to improve, we in the project
need to be the ones responsible for providing that experience.</p>
I'm not suggesting that we start writing debuggers from scratch. Instead I'm
suggesting we start packaging and distributing existing debuggers for all
platforms, together with plugins which extend those debuggers with Rust-specific
functionality. This will enable us to teach those debuggers about things like
traits, enums, and even async 3</a></sup>. And we can ensure that the debuggers +
plugins are updated regularly, and work correctly. Just like with LLVM, the
specific debugger should become just an implementation detail.</p>
^{3</sup>
We can work on ways to extend this to work with ecosystem
runtimes as well.</p>
</div>}Which debuggers to package 4</a></sup>, and how to distribute them are details
I'm confident we can figure out. We've done this before for other tools, so that
shouldn't be too different. The main question is how to expose the debugger, and
for that I would propose we start with the Debug Adapter Protocol (DAP)</a>,
LSP's less-famous sibling. Just like we use LSP to provide IDE features in an
IDE-agnostic way, we can use DAP to provide debugger support in an IDE-agnostic
way.</p>
^{4</sup>
Maybe we don't even want to package certain platform-specific
debuggers, and instead we just want to distribute bindings. Maybe different
platforms should make different decisions. Or perhaps people should be able to
choose. These are the kinds of details I expect experts to want to weigh in on.</p>
</div>}I believe by exposing the debugger as a server, we can enable others to
write tooling for it. For example: we could write standalone VSCode extensions which integrate
with it, or perhaps contribute a DAP integration directly to Rust-Analyzer
5</a></sup>.  Perhaps some people prefer an editor-based debugger flow; which
should be possible to build against DAP. And should DAP provide limitations, we
can always write extensions with the intent to later upstream them - which is
again something we do with LSP as well.</p>
^{5</sup>
I defer to the Rust-Analyzer team to comment on what the best course
of action here might be.</p>
</div>}A really interesting idea in the Rust debugger space has been the approach of plugging
a MIR interpreter into a debugger as an extension, which is then able to evaluate
Debug</code> impls at runtime. This has been prototyped</a> and
shown to be possible. In my opinion that's the ideal solution because it
doesn't require any user code to be changed, and it's also exactly the kind of
thing that the Rust project can do which external debugger projects can't. Just
write your Debug</code> impls, and the debugger will figure out how to interpret
them.  Making this practical will not be without challenges, but those are
mostly technical - and technical talent is something we're not short on in the
Rust project.</p>
Conclusion</h2>
Those are my thoughts on how I think we can structurally improve the debugging
user experience in Rust. I believe that by making the Rust project responsible
for the debugger experience we can provide a fundamentally better experience
than any external debuggers ever could. And I would be excited if this was
something the Debugging WG</a> would seek to pursue.</p>
One last note before publishing: Debugger extensions in VS Code also have
support for profiling and even REPL-like evaluation</a>. I think this is
part of DAP too, or some extension to it at least. I feel very similar about
profiling like I do about debugging: the Rust project should be responsible for
providing a premier user experience. But we have to start somewhere, and I think
if we can build the integrations needed to support debugging, we can eventually
grow the scope to become responsible for profiling too. Go</a> and JS</a> have this
already, and I'd love for Rust to catch up.</em></p>
Thanks to Michael Woerister</a> for reviewing!</em></p>

State Machines III: Type States
Mon, 02 Jan 2023 00:00:00 +0000
I've been writing a bit about state machines recently. In my last post I
explained how we could potentially leverage arbitrary self-types and anonymous
enums to support type-level state machines as a first-class construct. Which is
cool, but it had some readers asking: "How does this improve on the existing
type state pattern?"</em> Which is fair!</p>
Well, to start off by answering that question: it's only after authoring that
post that I learned that "type states" have a name! I was familiar with the
pattern, but I didn't realize it had a name. If you'd like to learn more about
them, Will Chrichton gave an excellent presentation about API design which also
covers type states</a>.</p>
Though type states are powerful, they're not the easiest to use - and can be
limited in expressivity at times. In this post I want to show what some of those
limitations are, and how we can over come them by encoding them using
first-class type-level state machines instead.</p>
editor's note: I tried capping this post off at 500 words and failed. I'm
sorry, here's 2500 words instead.</em></p>
Setting the stage</h2>
We'll continue using the example we've been using in the past two posts; a
traffic light which can transition between states. We'll take the simpler
variant of that for this post:</p>

green can transition to yellow</li>
yellow can transition to red</li>
red can transition to green</li>
</ul>
If we want to encode it with the type-state pattern, we have to create separate
structs for each of the states, and then parameterize the traffic light struct
over the state. This is roughly what that looks like
(playground</a>):</p>
use </span>std::marker::PhantomData;
</span>
</span>struct </span>Green;
</span>struct </span>Yellow;
</span>struct </span>Red;
</span>
</span>struct </span>TrafficLight<S> {
</span>    </span>_phantom</span>: PhantomData<S>
</span>}
</span>
</span>impl </span>TrafficLight<Green> {
</span>    </span>fn </span>new</span>() -> TrafficLight<Green> {
</span>        TrafficLight { _phantom: PhantomData }
</span>    }
</span>}
</span>
</span>impl </span>TrafficLight<Green> {
</span>    </span>fn </span>next</span>(</span>self</span>) -> TrafficLight<Yellow> {
</span>        TrafficLight { _phantom: PhantomData }
</span>    }
</span>}
</span>
</span>
</span>impl </span>TrafficLight<Yellow> {
</span>    </span>fn </span>next</span>(</span>self</span>) -> TrafficLight<Red> {
</span>        TrafficLight { _phantom: PhantomData }
</span>    }
</span>}
</span>
</span>impl </span>TrafficLight<Red> {
</span>    </span>fn </span>next</span>(</span>self</span>) -> TrafficLight<Green> {
</span>        TrafficLight { _phantom: PhantomData }
</span>    }
</span>}
</span>
</span>fn </span>main</span>() {
</span>    </span>let</span> light = TrafficLight::new(); </span>// TrafficLight<Green>
</span>    </span>let</span> light = light.</span>next</span>();        </span>// TrafficLight<Yellow>
</span>    </span>let</span> light = light.</span>next</span>();        </span>// TrafficLight<Red>
</span>    </span>let</span> light = light.</span>next</span>();        </span>// TrafficLight<Green>
</span>}
</span></code></pre>
This makes it so that if we attempt to call to_red</code> when we have a
TrafficLight<Green></code>, the compiler will disallow it. Type states allow us to
encode program invariants into the type system, preventing it from causing
runtime bugs.</p>
Where type states run into trouble</h2>
Type states become tricky when you start wanting to combine them with runtime
logic. After all, a traffic light example is nice: but what if we're actually
trying to run it? Take for example the following code, which can't be neatly
represented using type states:</p>
use </span>futures_lite::prelude::*;
</span>use </span>futures_time::prelude::*;
</span>use </span>futures_time::time::Duration;
</span>use </span>futures_time::stream;
</span>
</span>let mut</span> light = TrafficLight::new();
</span>let mut</span> interval = stream::interval(Duration::from_secs(</span>20</span>))
</span>while let </span>Some(_) = interval.</span>next</span>().await {
</span>    light = light.</span>next</span>(); </span>// Cycle to the next state every 20ms
</span>}
</span></code></pre>
We want to switch to the next state</code> every 20s, but this won't compile because
TrafficLight<Green></code> and TrafficLight<Yellow></code> aren't the same type, so they
can't be re-assigned to the same value. In order to do so we can match the
type states back to an actual enum, and wrap the type-state variant in it. This
could look something like this (playground</a>):</p>
use </span>futures_lite::prelude::*;
</span>use </span>futures_time::prelude::*;
</span>use </span>futures_time::time::Duration;
</span>use </span>futures_time::stream;
</span>
</span>enum </span>TrafficLightWrapper {
</span>    Green(TrafficLight<Green>),
</span>    Yellow(TrafficLight<Yellow>),
</span>    Red(TrafficLight<Red>),
</span>}
</span>
</span>impl </span>TrafficLightWrapper {
</span>    </span>pub fn </span>new</span>() -> </span>Self </span>{
</span>        TrafficLightWrapper::Green(TrafficLight::new())
</span>    }
</span>    
</span>    </span>pub fn </span>next</span>(</span>self</span>) -> </span>Self </span>{
</span>        </span>match </span>self </span>{
</span>            TrafficLightWrapper::Green(inner) => TrafficLightWrapper::Yellow(inner.</span>next</span>()),
</span>            TrafficLightWrapper::Yellow(inner) => TrafficLightWrapper::Red(inner.</span>next</span>()),
</span>            TrafficLightWrapper::Red(inner) => TrafficLightWrapper::Green(inner.</span>next</span>()),
</span>        }
</span>    }
</span>}
</span>
</span>let mut</span> light = TrafficLightWrapper::new();
</span>let mut</span> interval = stream::interval(Duration::from_secs(</span>20</span>))
</span>while let </span>Some(_) = interval.</span>next</span>().await {
</span>    light = light.</span>next</span>();
</span>}
</span></code></pre>
This is a non-trivial amount of boilerplate, just to work with code which uses
the type state pattern at runtime. 50 lines to encode just 3 states is probably
too much to be practical for most uses, even if the resulting API is flexible enough
to be used in a variety of scenarios. We should be able to do better than this!</p>
Doing better with enums</h2>
Now, say we had access to
enum-variant-types</code></a> (enum
variants as first-class types), and
arbitrary-self-types</code></a>, we
could imagine a reformulation of the traffic light as follows:</p>
//! A basic traffic light state machine
</span>//! Novel language features: enum-variant-types, arbitrary-self-types
</span>
</span>enum </span>TrafficLight {
</span>    Green,
</span>    Yellow,
</span>    Red,
</span>}
</span>
</span>impl </span>TrafficLight {
</span>    </span>pub fn </span>new</span>() -> </span>Self::</span>Green {
</span>        </span>Self</span>::Green
</span>    }
</span>
</span>    </span>pub fn </span>next</span>(</span>self</span>: </span>Self::</span>Green) -> </span>Self::</span>Yellow {
</span>        </span>Self</span>::Yellow
</span>    }
</span>
</span>    </span>pub fn </span>next</span>(</span>self</span>: </span>Self::</span>Yellow) -> </span>Self::</span>Red {
</span>        </span>Self</span>::Red
</span>    }
</span>
</span>    </span>pub fn </span>next</span>(</span>self</span>: </span>Self::</span>Red) -> </span>Self::</span>Green {
</span>        </span>Self</span>::Green
</span>    }
</span>}
</span>
</span>fn </span>main</span>() {
</span>    </span>let mut</span> light = TrafficLight::new(); </span>// TrafficLight<Green>
</span>    light = light.</span>next</span>();                </span>// TrafficLight<Yellow>
</span>    light = light.</span>next</span>();                </span>// TrafficLight<Red>
</span>    light = light.</span>next</span>();                </span>// TrafficLight<Green>
</span>}
</span></code></pre>
Because TrafficLight</code> is now a single type, it can be assigned to the same
variable without a problem. Paving over some details, we could imagine the
following could then work as well:</p>
// ..imports..
</span>
</span>let mut</span> light = TrafficLight::new();
</span>let mut</span> interval = stream::interval(Duration::from_secs(</span>20</span>))
</span>while let </span>Some(_) = interval.</span>next</span>().await {
</span>    light = light.</span>next</span>(); </span>// Cycle to the next state every 20ms
</span>}
</span></code></pre>
And that, in a nutshell, is why I believe that access to first-class state
machines at the type level would be helpful. It would enable us to use type
states in more places, which in turn would allow us to create more robust
abstractions, which yet in turn lead to fewer bugs.</p>
Embedding in structs</h2>
Being able to treat enums as type-level state machines has other benefits as
well: it makes it so when you embed a state machine in another type, the state
itself doesn't necessarily need to be observable from the outside either. Take
the following code:</p>
struct </span>Green;
</span>struct </span>Yellow;
</span>struct </span>Red;
</span>struct </span>TrafficLight<S> {
</span>    </span>_phantom</span>: PhantomData<S>
</span>}
</span>
</span>// A new struct containing multiple traffic lights
</span>struct </span>Crosswalk<S1, S2> {
</span>    </span>light1</span>: TrafficLight<S1>,
</span>    </span>light2</span>: TrafficLight<S2>,
</span>}
</span>
</span>impl</span><S1, S2> Crosswalk<S1, S2> {
</span>    </span>pub</span> async </span>fn </span>run</span>(</span>self</span>) {
</span>        </span>// loop here.
</span>    }
</span>}
</span></code></pre>
The internal states for each of the traffic lights necessarily becomes externally
observable, even if no external methods ever need to observe it. The internal
state leaks</em> to the external API surface, making it a leaky abstraction. We
could of course define the wrapper type we saw earlier again. But say we
did have first-class state machines, we could encode it directly like this instead:</p>
enum </span>TrafficLight {
</span>    Green,
</span>    Yellow,
</span>    Red,
</span>}
</span>
</span>// A new struct containing multiple traffic lights
</span>struct </span>Crosswalk {
</span>    </span>light1</span>: TrafficLight,
</span>    </span>light2</span>: TrafficLight,
</span>}
</span>
</span>impl </span>CrossWalk {
</span>    </span>pub</span> async </span>fn </span>run</span>(</span>self</span>) {
</span>        </span>// loop here.
</span>    }
</span>}
</span></code></pre>
We could still choose to surface the enums to the outside world if we wanted to
(more on that later in this post). But there is a real benefit to not having to
if you don't want to, without needing to define pages of boiletplate!</p>
Enum method unification</h2>
One assumption I've made is that if you have an enum with three states, and you
define a method for all three states, then that method should become available
even if we don't know which state we're in.</p>
Using match</code> we can always find out what state we're in by inspecting the
enum's tag, meaning we can always figure out what state we're in and what state
we should transition to, at runtime.</p>
This also means that, say, if a method is only available for 2/3 of the states,
as long as we can match</code> down to those states, the method should be available:</p>
enum </span>RGB { Red, Green, Blue }
</span>
</span>impl </span>RGB {
</span>    </span>fn </span>test</span>(</span>self</span>: </span>Self::</span>Green | </span>Self::</span>Blue) {
</span>        println!("</span>can print!</span>");
</span>    }
</span>}
</span>
</span>fn </span>try_to_print</span>(</span>rgb</span>: RGB) {
</span>    </span>match</span> rgb {
</span>        </span>RGB</span>::Green | </span>RGB</span>::Blue => rgb.</span>print</span>(),  </span>// `print` is defined for both states here
</span>        _ => { </span>/* do nothing */ </span>}
</span>    }
</span>}
</span></code></pre>
Enums as type params</h2>
Even if we could hide the internal state sometimes, doesn't mean we'd always
want to do it. I think one other benefit of "enums as state machines" in this
model would be just to improve the existing type-state pattern. Take for example
my tasky</code></a> crate. In v3.0.0 we
distinguished between a "local builder" and "non-local builder" at the type
level. This was needed so we could add different IntoFuture</code> implementations to
them. This is how they're defined:</p>
/// Sealed trait to determine what type of builder we got.
</span>mod </span>sealed {
</span>    </span>pub trait </span>Kind {}
</span>}
</span>
</span>/// A local builder.
</span>pub struct </span>Local;
</span>impl </span>sealed::Kind </span>for </span>Local {}
</span>
</span>/// A nonlocal builder.
</span>pub struct </span>NonLocal;
</span>impl </span>sealed::Kind </span>for </span>NonLocal {}
</span>
</span>/// Task builder that configures the settings of a new task.
</span>pub struct </span>Builder<Fut: Future, K: sealed::Kind> {
</span>    </span>kind</span>: PhantomData<K>,
</span>    </span>future</span>: Fut,
</span>    </span>builder</span>: ...,
</span>}
</span></code></pre>
That's a lot of work to encode the state 1</a></sup>. And just from glancing over the
rustdoc output you'd never know that that's what this is for. Instead it'd be
far nicer if we could define typestates using enums, and not actually have to
bother with sealed traits and disjointed structs:</p>
^{1</sup>
A note on sealed traits: this is the first time in the post we're using
them, but they're useful to ensure that only types defined within the crate can
ever be passed to it. If we had support for concrete types in bounds + enums as
first-class types, then we'd probably have less need for sealed traits to be
used in this way.</p>
</div>}enum </span>BuilderKind {
</span>    Local,
</span>    NonLocal,
</span>}
</span>
</span>/// Task builder that configures the settings of a new task.
</span>#[</span>derive</span>(Debug)]
</span>pub struct </span>Builder<Fut: Future, K: BuilderKind> {
</span>    </span>kind</span>: K,
</span>    </span>future</span>: Fut,
</span>    </span>builder</span>: ...,
</span>}
</span></code></pre>
I believe doing this would require us to enable using concrete types as
type-bounds. I believe this is something which comes up repeatedly, and we do
want to resolve eventually for other reasons as well. Perhaps "better
typestates" could be added to that list as well now?</p>
Enum variant tracking</h2>
Another feature which would probably be helpful here would be something I call
"enum variant tracking". Take for example the following code, which inserts a
value into Option</code> if it doesn't exist, and then provides us with mutable
access to it:</p>
let mut</span> option = None;
</span>if</span> option.</span>is_none</span>() {
</span>    option = Some(</span>12</span>);
</span>}
</span>let</span> num: &</span>mut u32 </span>= option.</span>as_mut</span>().</span>unwrap</span>(); </span>// may panic
</span></code></pre>
Sure, Option::unwrap_or_else</code> exists and does the same. But assume for a second
we've defined our own Option</code>, and we need to implement our own. We should
never</em> have to unwrap</code> here, since we know from the code that we're
guaranteed</em> to always have a Some</code> variant here. And the is_none</code> check is
redundant here, since we're guaranteed to always have a None</code> variant to start
with.</p>
The idea is that if we know</em> for a fact that we're dealing with a specific
variant, we should be able to skip the ceremony and checks to access that
specific variant. Instead of just knowing we have an Option</code>, with "enum
variant types" we should be able to accurately track that we in fact start with an
Option::None</code>, which always becomes Option::Some</code> after. The latter might be a
bit more tricky, but given both option</code> and is_some</code> are const</code>, we could
imagine that the compiler should be able to always know this.</p>
To clarify though: while I think this feature might make sense from a
theoretical perspective, it may not make sense from a practical one</em>. I'm not
trying to argue we should or shouldn't support this, more that it might be
within the realm of possibility if we get "enum variant types". This
feature would also only be intended for local reasoning only. Nothing should
ever cross function boundaries; other features should be available to enable us
to more granularly annotate things. I think if we ever start to get serious
about wanting to allow people to write panic-free code, this may become
interesting. I'm kind of reminded of the (unstable)
Error::into_ok</code></a>
method, which is gated behind the unwrap_infallible</code> feature on nightly.</p>
let mut</span> option = None;
</span>if</span> option.</span>is_none</span>() {
</span>    option = Some(</span>12</span>);
</span>}
</span>let</span> num: &</span>mut u32 </span>= option.</span>as_mut</span>().</span>into</span>(); </span>// will never panic
</span></code></pre>
narrowing of state modifications</h2>
Something we haven't yet covered, but probably should is the fact that all the
state transition things we've talked about only work with owned</em> types. They
don't at all work with mutable references. When we're calling next</code> on an
iterator we get a &mut Self</code> type. When transitioning between state machines,
wouldn't it make sense for the state transitions to be able to mutate
Self</code> rather than always having to return a value?</p>
I've saved this one for last, because this is where I really start to hesitate
to be honest. All the features we've covered so far are in my opinion fairly
natural extensions to what we already have. Subsets of enums, being able to use
enums in more places, being able to narrow our views. What we're talking about
here feels much further out there</em>, and so I'm also way less sure whether it's a
good idea.</p>
The basic premise is as follows, we want to enable the following to work:</p>
fn </span>main</span>() {
</span>    </span>let mut</span> light = TrafficLight::new(); </span>// TrafficLight<Green>
</span>    light.</span>next</span>();                        </span>// TrafficLight<Yellow>
</span>    light.</span>next</span>();                        </span>// TrafficLight<Red>
</span>    light.</span>next</span>();                        </span>// TrafficLight<Green>
</span>}
</span></code></pre>
This means we'd need to find a syntax in the view type to not only describe what
types go in, but also what the type will transition to. Niko's view type post
didn't cover this, and my previous state machine post didn't cover this either.
But at least conceptually, it seems to me that if we would enable a narrowing of
function parameters and function return types, we might also want to provide a
way to express a narrowing of value modifications.</p>
I'm completely just riffing here, but maybe something like this? This is
supposed to indicate that after the function returns, the value of self</code> will
be Self::Yellow</code>.</p>
impl </span>TrafficLight {
</span>    </span>pub fn </span>next</span>(</span>self</span>: </span>Self::</span>Green -> </span>Self::</span>Yellow) {
</span>        </span>Self</span>::Yellow
</span>    }
</span>}
</span></code></pre>
I'm really unsure about this. But I think if we end up doing anonymous enums
and view types, then this might be a question we may want to find an answer for.</p>
Language feature overview</h2>
To recap: here is the list of language features we've covered in this series so
far. None of them are quite enough to bring us all the way, but if we combine
them they might unlock the ability to improve the status quo of type states:</p>

enum variant types</strong>: Enable reasoning about enum members as first-class
types. We can't specify say that a method returns a specific enum member. We can
only ever return the full enum.</li>
arbitrary self-types / enum view-types</strong>: Enable declaring Self: SomeEnum::Member</code>, allowing us to declare methods which are only available for
certain enum variants.</li>
anonymous enums / enum subsets</strong>: To enable us to declare sub-sets of enums using syntax
like: Member1 | Member2</code>. For example, we may want to declare that a certain
method can only return a specific sub-set of errors, or specific network status codes.</li>
enum member unification</strong>: Say a method next</code> is declared for
SomeEnum::Blue</code> and SomeEnum::Red</code>. If we obtain Blue | Red</code> then we should
be able to call that method in either case. From the perspective of the language
it should be no different than if we'd have an enum with just those two members,
and we'd have the method implemented for that entire enum.</li>
enum variant tracking</strong>: If we know</em> for a fact that we're dealing with a
specific variant, we should be able to skip the ceremony and checks to access
that specific variant.</li>
narrowing of state modifications</strong>: If we end up with anonymous enums and
view types, we'll have syntax to express narrowings of both owned input and
output function params. If that's the case, we may also find ourselves wanting
to express narrowings of in-out params, which requires a way to express.</li>
concrete type bounds</strong>: So we can use where S: SomeEnum::Member</code> directly as a bound,
where State</code> is an enum. Currently using sealed traits + individual structs
seems like the best way to approximate this.</li>
</ul>
Conclusion</h2>
We've covered quite a number of language features so far. Just to be clear: I'm
not trying to say that we should be implementing any of this, and even if we
decided to: that we should be prioritizing this. I'm more interested in thinking
out loud about how we could improve on type states, and summarizing which
language features could be implemented to bring us closer to that.</p>
With that in mind, I hope this made for an interesting read. Thanks for reading,
and happy new year!</p>
Thanks to Vagrant</a> for proof reading this post!</em></p>

2022 in review
Fri, 23 Dec 2022 00:00:00 +0000
It's pretty quiet at work right now, and every other application on my phone is
telling me it's time to look back. So why not lean in and look back at the last
12 months. Not some big reflection time, but to share some things I've done and
liked. I've published about a novel's worth of work in the past year, and
probably read a multitude of that. So if you're looking to read some tech
related things over the holidays, maybe you'll find something fun in this list!</p>
My own work</h2>
Here are things I've worked on and written about this year:</p>
Rust Language Work</h3>

Announcing the Keyword Generics
Initiative</a>:
Where we announce the creation of the "keyword generics initiative", a new
group under the language team looking to add a new type of generic to the language 1</a></sup>.</li>
More Enum Types</a>: We discuss
what potentially the enum-equivalent of tuples and impl trait</code> could look like. This could be useful, for say, easier application error handling.</li>
State Machines II</a>: We take a
look at typestates2</a></sup>  in Rust, and explore a way to make them a first-class
citizen in Rust by leveraging arbitrary-self-types, enums as first-class
types, and anonymous enums. The biggest innovation here is that we'd have a
way to prevent them from leaking to enclosing types by using enums instead of generics.</li>
Inline Crates</a>: We discuss
the difference between "crates" and "modules", and explore what feature parity
between the two could look like, and what challenges enabling that would bring.</li>
Keywords I: Unsafe Syntax</a>:
Reflecting on the various meanings of the unsafe</code> keyword in Rust.</li>
Keywords II: Const Syntax</a>:
Reflecting on the various meanings of the const</code> keyword in Rust.</li>
</ul>
^{2</sup>
I didn't know "type states" had a name until after I wrote the post, so
they're not mentioned by name in the post.</p>
</div>}^{1</sup>
I'm planning to write an update on the work we've done so far after
the holidays. Stay tuned for that I guess?</p>
</div>}Async Work</h3>

Why Async Rust</a>: In which I
explain what async Rust is, which features it provides, and how we can
meaningfully talk about whether it makes sense to adopt.</li>
Futures Concurrency III:
Select!</a>: A deep-dive into
the async select!</code> macro, discussing its various shortcoming, and looking at
an alternative structured concurrency operator which doesn't share these
shortcomings.</li>
Safe pin projections through view
types</a>: Goes
into some detail on how we could leverage the "view types" proposal to make
pin projections safe. Not quite a full proposal, but intended to give a first
impression of what this might look like.</li>
Futures Concurrency IV: Join
Ergonomics</a>: We take a
look at Swift's async let</code> feature, and how we can enable similar
semantics in Rust's libraries using IntoFuture</code>.</li>
Enhancing .await</code> with
IntoFuture</code></a>:
The release 1.64 release notes cover the stabilization of IntoFuture</code>, and
show some cool things you can do with it.</li>
Async-Iterator crate</a>: A
short PSA that I tried the unstable "async fn in traits" feature, and
published a crate which makes use of it.</li>
Postfix Spawn</a>: A first look at
the nature of async Rust's spawn</code>-family of APIs, and how we can make them
behave in a structured manner. I wrote this in a fever-dream, and probably
should write a follow-up.</li>
Async Cancellation II: Time and
Signals</a>: A first look at
a newly created suite of time-based APIs, combining concurrency and
cancellation to create a cohesive set of async control flow constructs.</li>
</ul>
Miscellaneous</h3>

Rust 2023</a>: My thoughts and vision
on the Rust project, late 2022 edition.</li>
</ul>
Other's work</h2>
Here are things I've read this year which were a highlight for me. Some of it
was published this year, other things are from earlier years. But either way,
these were highlights for me:</p>
Rust</h3>

"Memory Safe Languages in Android 13" by Vander
Stoep (Google)</a>:
provides compelling evidence that adopting Rust does in fact meaningfully reduce the number
of memory safety bugs in codebases. And crucially: significantly reduces the
number of high/critical vulnerabilities. Imo this is one of the most important
reports written about Rust in recent years.</li>
"Rust Atomics and Locks" by Mara Bos</a>: While
technically not yet published, I did get a chance to read the pre-release version,
and go in-depth on some of the topics in the book with Mara over the past
year. I haven't yet finished the book, but I already feel like I understand
atomics so much better now (read: at all), and in my opinion this is a
must-have for everyone looking to go going beyond the basics in Rust.</li>
"Type-Driven API Design in Rust (video)" by Will
Crichton</a>: walks through
designing an API in Rust step-by-step, showing how Rust's type system
can be leveraged to guard against increasingly more kinds of invalid uses.</li>
"Async destructors, async genericity and completion futures" by Sabrina
Jewson</a>: a foray into the design
challenges of designing an "async Drop" feature in Rust, providing some sharp
insights on the matter. Deep-dives into the design problem space as done at
the start of this post are in my opinion something we need more of in the Rust
project.</li>
"The Tower of Weakenings: Memory Models for Everyone" by Aria
Beingessner</a>: Explains what
provenance is, why we should care, and how we can bring strict provenance to
Rust. This is one of those posts which really peels back the cover on the
sticks-and-bubblegum nature we're building on, and then proposes a way to
actually fix that without proposing everyone need become an expert in the process.</li>
</ul>
Capabilities</h3>

"No Ghosts!" by Sunfishcode</a>:
proposes and explores a design principle for components in complex software
systems.</li>
"What is a Capability" by
Sunfishcode</a>: Answers the
question: "What is a capability?".</li>
"Capabilities: effects for
free" by Craig et al.</a>: bridges
the concept of "capabilities" with "effects", showing how we can use effects
as capabilities.</li>
"Capabilities for Resources and Effects" by
Martin Odersky</a>:
The Scala project got a 7-year EU research grant to investigate adding capabilities and
effects into the language. This describes what that work will be focusing on.</li>
"Contexts and Capabilities in Rust" by Tyler
Mandry</a>:
(I was involved with this), describes the "context problem" in Rust, and
explains how we could solve this by adding a new "with-clause" language feature.</li>
"The Path to Components (video)" by Luke
Wagner</a>: Goes in-depth on the
the WASM component model: what it is, where it's currently at, and where it's
headed.</li>
</ul>
Systemic Safety</h3>

"Blaming human error for an outage is like blaming gravity for a building
falling over" — Tef on Twitter</a></p>
</blockquote>

"The New View of Safety (video)" by Todd
Conklin</a>: "The next time
something goes wrong, switch you thinking from 'who failed' to 'what failed'."
Incredibly entertaining speaker talking at the United Steel Workers conference
on health safety and environment. It's not directly about tech, but highly
applicable to tech.</li>
"The Field Guide to Understanding Human
Error" by Sidney Dekker</a>:
I've started this and plan to finish it over the holidays. But so far it's
proben to be an incredibly useful resource reflecting on what failure is, and
how to think about it.</li>
"How Complex Systems Fail" by Richard
Cook</a>: A great overview of how to think
about failure, linking to relevant research backing up each point. Authored by
a medical doctor, but highly relevant to many fields including software
engineering.</li>
"Confessions of a Recovering Engineer: Transportation for a Strong
Town"</a>:
An interesting dive into the world of transportation, how to think about
safety, and provides a perspective on the politics of institutionalized
engineering.</li>
"Crash-Only Software" by Candea et
al.</a>: Another
classic computer science paper which I'd only ever heard about, but never
actually read. It provides interesting perspectives on failure modes in
computing, and how to deal with them.</li>
</ul>
Compilers</h3>

"Cranelift Progress in 2022" by Chris
Fallin</a>: As
someone who likes compiler theory more than they enjoy working on compilers
directly, this post is a dream. It covers all the compiler improvements that
have happened on Cranelift over the past year. It's super inspiring to see a
production-grade project have such a strong focus on correctness and
meaningfully better abstractions. Being able to follow this work in the open is
such a treat!</li>
"A catalogue of optimizing transformations" by
Allen et al.</a>:
I've seen this paper referenced many times, but I didn't actually read it
until this year. It's pretty pragmatic in how it covers the main
optimizations a compiler can implement. Folks sometimes refer to this as: "the
good ones", as just by implementing these you can get like 80% of the way there.</li>
</ul>
Miscellaneous</h3>

"What we owe each other" by Vagrant
Gautam</a>: A
really good post on justice, feelings, and accountability. Being a human can be
hard sometimes, and Vagrant does a fantastic job navigating this. Not an easy
read, but one that I found worthwhile.</li>
"The Five Minute Feedback Fix" by Hillel
Wayne</a>: shares
a practical formal verification technique which takes all of five minutes to
learn, and can be applied to virtually any language or project.</li>
"Bullshit Jobs" by David
Graeber</a>: A reflection on the
nature of work, which felt relevant to me since, y'know, I'm a worker and all.</li>
"IETF RFC 9110: HTTP Semantics" by Fielding et
al.</a>: The
IETF published a new series of HTTP specs this year, unifying the myriad of
previous documents into "semantics", "caching", and separate documents
for HTTP/1.1, HTTP/2, and HTTP/3. The "semantics" document is fantastic
if you just want to learn how HTTP works, without going into the exact details
of the various backing wire protocols. I'm a big fan!</li>
</ul>
Conclusion</h2>
I hope some of this was interesting. I'm off to finish reading some of the books
on this list. Happy holidays!</p>

Rust 2023 (by Yosh)
Wed, 21 Dec 2022 00:00:00 +0000
Previous posts1</a></sup>: 2018</a>, 2019</a>, 2020</a>, 2021</a></em></p>
^{1</sup>
I didn't write a "Rust 2022" post, sorry.</p>
</div>}The core team used to put out a yearly call for blog posts. My colleage Nick
published their "Rust in 2023"</a>
post last week, and encouraged others to do the same. I like the idea of taking
a moment to reflect on larger topics, and so well, why not write a post!</p>
I want to do this a bit differently from the usual formula though. Rather than
writing something with the specific intent to build some sort of "Rust 2023
roadmap", I want to instead take this as an opportunity to reflect on the state,
values, and priorities of the Rust project. More of a snapshot of my current
perspectives, than a concrete list of action items I think should be tackled.
Here goes!</p>
Async</h2>
Probably most of my work will continue to focus on async Rust. A majority of
programming jobs are either frontend or backend related 2</a></sup>, and async
is important to both. Anecdotally I know that async has been a major
driver for the adoption of Rust - and the more we can improve async Rust, likely
the better we'll do in those fields.</p>
^2</sup>The 2022 stackoverflow survey notes 26% of people work on front-end,
and 43% of people do backend work. These numbers are non-exclusive, since folks
could be doing both
(src</a>) - but I think
it's fair to categorize this as a majority of jobs.</p>
</div>
Work has been progressing a lot here. Diagnostics have become significantly
better. We now have IntoFuture</code> stabilized. And both safe stack-pinning and
async fns in traits are on the horizon. And slightly further out is the work on
async iteration, async drop, async main, and keyword generics.</p>
I should probably get around to writing the end-of-year update post for the
Async WG I promised I would write. But the tldr is that async Rust is important
to Rust's future, we've made a lot of progress over the past year - and I think
we're in a pretty good spot to keep making progress.</p>
Ambition</h2>
It's winter holidays, and I'm not going to ping people to clear quotes. But
someone I respect a lot once remarked: "Rust does best when we're ambitious"</em> -
and I agree strongly with this. I think the work we're doing on Rust is far from
done. Rust is in a spot where people can be productive, but there are still so
many limitations. And when people run into this, I want us to do better than:
"Well, that's just the way things are". We can in fact have nice things!</p>
As a general attitude, I want us to stay ambitious. To imagine a world in which
better things are in fact possible. And then doing the work to get there.  Rust
combines the desire for a radically better programming language with the
motivation to actually get things in people's hands. And I think our work is
some ways only just getting started 3</a></sup>.</p>
^{3</sup>
It's hard to say how long programming languages are expected to
last, but we're seeing Rust adopted in some pretty high-stakes places which are
likely to stay around for a very long time</em>. This means it's possible Rust will
outlast us, meaning that in retrospect now might very well still be "the early
days" in terms of Rust adoption. We won't know for sure until we're there, but
imo this is not an unlikely outcome of our current trajectory.</p>
</div>}^{4</sup>
I feel like everyone attributes this quote to someone else at this
point, but: the design space of a programming language is fractal! There are
infinite details to dig into, and there's also a real art in deciding what to
prioritize.</p>
</div>}Cohesion</h2>
"What is Rust" is one of those fun open-ended questions which doesn't have a
right answer. After all: Rust is a programming language. Or maybe Rust is a
feeling in our hearts? I think Rust is the friends we make along the way.
Really, it's us who get to decide what Rust means to us.</p>
Pragmatically, I think Rust is not just</em> a programming language. Cargo
is Rust. Rustup is Rust. Crates.io is Rust. All the tools,
guides, websites, people, and programs together make "Rust". And I think that
with that come some interesting questions about governance, power, and cohesion
as well.</p>
I won't bore you with some long essay on the reflection of power dynamics within
open source ecosystems. But what I will say is that Rust has done remarkably
well in ensuring people using the language are presented with a relatively
cohesive experience! The compiler, formatter, versioning tool, guide, package
manager, and standard library are all owned and operated by the Rust project.
And I think this is a strength we should continue to build on.</p>
I'm really happy that we've recently seen the addition of the cargo add</code>
command</a>, removing the need to rely on the external (yet trusted) cargo-edit</code>
crate. And if I'm not mistaken work on
cargo-semver-checks</a> is
currently ongoing with the intent to integrate it into mainline cargo as well. 5</a></sup></p>
^5</sup>Somewhat proudly one of my own contributions along these lines in
the past year has been to bring the core functionality of the num_cpus</code> crate
into the
stdlib</a>. One
less crate to learn about!</p>
</div>
I believe that one of the main strengths of Rust is that we're creating
something together which looks and feels cohesive. I think it's good to keep
adding features as long they end up feeling part of a whole. And I think this is
something that's worth reflecting on, and identifying potential places where we
may be able to lean further into this.</p>
Automation</h2>
I sometimes think about the 80/20 rule: "80% of work takes 20% of time; the
remaining 20% in 80% of time." I'm not sure if this has a proper name, but
there's also what I like to call the 100% rule: "Once you've solved 100% of a
problem, you can stop thinking about it completely".</em></p>
Take for example memory safety in systems programming languages. Using fuzzers
and sanitizers you may be able to easily address 80% of bugs. But the remaining
20% of bugs will take some effort to find. And even when you do; code changes
over time, which means that you'll be unlikely to ever afford to stop looking
for these types of bugs.</p>
Compare that with the borrow checker, which guarantees there will never</em> be any
memory safety bugs: meaning you can catagorically eliminate the concern
alltogether. The 80/20 rule is often used to indicate that "80% of effort is
good enough". But I think it's important to realize that sometimes achieving 100%
of a solution can provide a value-add which far outweighs all incremental steps
that come before it.</p>
I want us to keep looking for ways in which we can structurally eliminate issues
through automation, so we can stop thinking about them alltogether. Taking
cargo-semver-checks</a> as an
example again: rather than needing to learn and check for things like
"variance" and "contravariance"</a>.
When publishing new versions of crates, we should just have a tool which checks
it for us - and patiently explains how we can fix the issues, even if we're not
completely sure about what the underlying rules are.</p>
I think it was Ralf Jung who once said something along the lines of: "The job of
an expert is to learn everything about a field there is to learn, and then
distill it so that others don't have to."</em> 6</a></sup>. And in my opinion
building tools is one of the best ways we have to propagate domain expertise.
I think this is something which we should keep leaning into 7</a></sup>.</p>
^{6</sup>
This was definitely a TWIR quote of the week at some point. I think
late 2020/early 2021 ish? If someone remembers the link I'll update this post!</p>
</div>}^7</sup>I believe programming language features such as the borrow
checker, or capabilities</a> fall
under this as well. Rather than memorizing and manually validating a set of
rules, we can/should use automation (a compiler) to automate this reasoning for us.
Because for example even interpreted languages are not free from having to
reason about lifetimes and ownership; the only difference is that the language
doesn't provide facilities to automate this reasoning. So people instead
have to rely on techniques such as defensive copying to prevent aliasing bugs from
occurring. (This could probably be a post on its own, so I'll just stop here hah.)</p>
</div>
Conclusion</h2>
I can probably keep talking about Rust at length; that's what this blog is for
after all. But I think these are some of the key points I wanted to hit. Before
closing out though, I probably should briefly cover my thoughts on governance:</p>

I'm happy folks are working on governance. I trust them and their work.</li>
I'm happy we're seeing different variants of "light weight RFCs" popping up.
I believe both compiler (MCP</a>) and libs (ACP</a>) have these now, and I'm excited
to give some of these a shot.</li>
Governance is fundamentally about relationships, and as long as the project
changes, so will governance. The work here will likely never be "done", and
that's probably just the nature of it.</li>
</ul>
To close out: I think we're doing great, and I'm excited for the direction
we're headed. Happy 2023!</p>

Async Iteration II: The Async Iterator Crate
Mon, 19 Dec 2022 00:00:00 +0000

Async Iteration I: Semantics</a></li>
Async Iteration II: Async Iterator Crate</a> (this post)</li>
Async Iteration III: Async Iterator Trait</a></li>
</ol>

Hey all, today was my first day back from PTO. While away, I was thinking what I
often think: I should write more on here. With Twitter effectively defunct, and
my Mastodon being set to ephemeral mode (posts auto-delete after 30 days), I
should start keeping a better record of what I'm up to somewhere more
permanent</em>. And this blog seems as a good a place as any.</p>
Async Iterator</h2>
I wanted to share that I've published a crate:
async-iterator</code></a>. Its
purpose was partly to test out the new "async fn in traits"</a> feature on
nightly. And partly to have a workable sketch of what we expect "async iterator"
to behave like.</p>
use </span>async_iterator::prelude::*;
</span>
</span>async </span>fn </span>foo</span>(</span>iter</span>: impl Iterator<Item = </span>u32</span>>) -> Vec<</span>u32</span>> {
</span>    iter.</span>collect</span>().await
</span>}
</span></code></pre>
This is not final</em> shape of the trait though, since we're working on keyword
generics</a> and expect to retroactively add the async Iterator</code> functionality to
the Iterator</code> trait which then becomes generic over "asyncness". But those
should just be the same semantics exposed through a different mechanism. Other
than that, the crate is pretty minimal, yet covers pretty much all the common
usage patterns:</p>

The base async Iterator</code> trait implemented through async fn next</code> instead of fn poll_next</code></li>
The ability to collect</code> into a vec</code>, using  actual async IntoIterator</code> and FromIterator</code> bounds</li>
The ability to async map</code> over values in the iterator</li>
The ability to extend</code> vec</code> with an async iterator</li>
</ul>
The only thing missing is async fn filter</code>, which borrows items and so needs
some form of "async closures" to work. But all in all this means it works! -
"async fn in traits" is real on nightly, and actually does what we want it to
do! I think that's really exciting! I don't think the async-iterator</code> crate
should be used in actual projects though. It's an interface which only works on
nightly, and interfaces only really work when they're uniformly adopted. And
between futures 0.3.0 and the stdlib I don't think we should migrate to any
in-between interfaces in the interim. But I do think it will be really useful to
write experiments against!</p>
Future Directions</h2>
I still have to check in with the "async fns in traits" team to see what they're
up to (being gone for a month is a long time). But something I might consider
doing in the new year is do a rewrite of the async-h1 crate</a> based on
"async fn in traits". The internals are basically a giant poll</code>-based state
machine, and we've theorized for a while now that rewriting it in terms of
async fn</code>s would significantly simplify its internals. It seems like this
should finally be possible, but I'll have to check with folks first whether
doing this will actually be helpful now as well.</p>

Keywords II: Const Syntax
Wed, 26 Oct 2022 00:00:00 +0000
On the Keyword Generics Initiative</a> we spend a lot of time talking about, well,
keyword generics! We're working on designing a system which works with Rust's
various effects</em>, which are represented through keywords. And sometimes we
discover interesting interactions or meanings of keywords.</p>
Previously I've written about the meaning of unsafe</code></a>, showing how it actually
is a pair of keywords whose meaning depends on whether it's in caller or
definition position. const</code> shares some similarities to this, and in this post
we'll be looking closer at the different uses of const</code>.</p>
always-const execution</h2>
If you want to execute a function during compilation, you need to have a way to
make that happen. This is what we've been referring to as an "always const"
execution, where you have a context which will always be evaluated at
compile-time and never during runtime.</p>
In Rust we can create an always-const context in a few different ways:</p>
// Defining a `const` value
</span>const </span>HELLO</span>: &</span>str </span>= {
</span>    "</span>hello</span>"
</span>};
</span>
</span>// Defining a const value in an array expression
</span>[</span>0</span>u8</span>; {</span>1 </span>+ </span>1</span>}];
</span>
</span>// Using `RFC 2920 experimental in-line const`
</span>let</span> x = </span>const </span>{ "</span>hello</span>" };
</span>
</span>// In const-generic parameters through `#![feature(`const_evaluatable_checked`)]`
</span>fn </span>foo</span><</span>const</span> N: </span>usize</span>>() -> [</span>u8</span>; N + </span>1</span>] { [</span>0</span>; N + </span>1</span>} }
</span></code></pre>
These contexts also disallow</em> the use of non-const values. Using them there
will quickly yield an error:</p>
fn </span>foo</span>(</span>length</span>: </span>usize</span>) {
</span>    [</span>0</span>u8</span>; length];
</span>}
</span></code></pre>
   Compiling playground v0.0.1 (/playground)
</span>error[E0435]: attempt to use a non-constant value in a constant
</span> --> src/main.rs:5:11
</span>  |
</span>4 | fn foo(length: usize) {
</span>  |        ------ this would need to be a `const`
</span>5 |     [0u8; length];
</span>  |           ^^^^^^
</span>
</span>For more information about this error, try `rustc --explain E0435`.
</span>error: could not compile `playground` due to previous error
</span></code></pre>
never-const execution</h2>
Code which cannot be evaluated during compilation is considered "never-const".
These contexts are effectively "runtime-only". Because const</code> is opt-in in
today's rust, it means that functions by default will not and cannot be
evaluated during compilation.</p>
fn </span>hello</span>() -> &</span>'static str </span>{
</span>    "</span>hello</span>"
</span>}
</span>
</span>const </span>HELLO</span>: &</span>str </span>= </span>hello</span>();
</span></code></pre>
error[E0015]: cannot call non-const fn `hello` in constants
</span> --> src/lib.rs:5:23
</span>  |
</span>5 | const HELLO: &str = hello();
</span>  |                     ^^^^^^^
</span>  |
</span>  = note: calls in constants are limited to constant functions, tuple structs and tuple variants
</span></code></pre>
maybe-const execution</h2>
The third type of const</code> context is "maybe-const". This is a context which can
be evaluated both during compilation and at runtime. The classic const fn</code>
definition is considered a "maybe-const" context:</p>
// Can be evaluated both at runtime and during `const` execution.
</span>const fn </span>square</span>(</span>num</span>: </span>u64</span>) -> </span>u64 </span>{
</span>    num * num
</span>}
</span>
</span>// Const evaluation
</span>const </span>NUM</span>: </span>u64 </span>= </span>square</span>(</span>12</span>);
</span>
</span>// Runtime evaluation
</span>fn </span>main</span>() {
</span>    println!("</span>{}</span>", </span>square</span>(</span>12</span>));
</span>}
</span></code></pre>
As an aside: In practice both these functions will be evaluated during compilation
thanks to an optimization called "const folding". But that's just an
optimization; not all code can be const-folded, and the code wouldn't compile if
it wasn't valid at runtime. Whether or not a function is marked const</code> has no
impact on whether or not the code will be const-folded. But I figured it'd at
least be worth mentioning the optimization as existing once in this post.</p>
const traits</h2>
Something which we don't yet have is the idea of "const traits". Right now you
can think of them like this:</p>
const trait </span>Trait {
</span>    </span>fn </span>func</span>() -> </span>usize</span>;
</span>}
</span></code></pre>
This is a "const trait" with a single "const method" which returns a type. Just
like const fn</code>, this should be considered a "maybe-const context": it should
both be able to be evaluated at runtime and during compilation. For example:</p>
// evaluates both at runtime and during compilation
</span>const fn </span>foo</span><T: </span>~</span>const</span> Trait>() -> </span>usize </span>{
</span>    T::func()
</span>}
</span></code></pre>
the need for always-const bounds</h2>
The current system of const</code> syntax has some interesting limitations. Take for
example the following code:</p>
// Can be called at runtime with a `T` which doesn't impl
</span>// `const Trait`.
</span>const fn </span>foo</span><T: </span>~</span>const</span> Trait>() {
</span>    [(); <T as Trait>::func()];
</span>    </span>//   ^ illegal
</span>}
</span></code></pre>
What we're seeing here is an "always-const" context nested in a "maybe-const"
context. A fun twist on the usual effect sandwich. In theory that should be
fine, but the issue here is that we're using a variable which is "maybe-const".
To get an understanding of why this is bad, here is roughly the same example,
but without the traits:</p>
const fn </span>foo</span>(</span>len</span>: </span>usize</span>) {
</span>    [(); len];
</span>}
</span></code></pre>
This yields the following error about wanting to have an "always const" value,
but what we got is a "maybe const" value:</p>
   Compiling playground v0.0.1 (/playground)
</span>error[E0435]: attempt to use a non-constant value in a constant
</span> --> src/lib.rs:2:9
</span>  |
</span>1 | const fn foo(len: usize) {
</span>  |              --- this would need to be a `const`
</span>2 |     [(); len];
</span>  |          ^^^
</span>
</span>For more information about this error, try `rustc --explain E0435`.
</span></code></pre>
The earlier trait example this would yield a very similar error, but using more
steps. Thinking about solutions here, we practically have two ways to design
our way out of this:</p>

Find a way to declare the maybe-const foo</code> context as always-const.</li>
Find a way to declare the maybe-const T: ~const Trait</code> as always-const.</li>
</ol>
This is an unresolved problem, so we're full on speculating here. But we could
imagine that if we support ~const Trait</code> meaning "maybe-const", we could also
support const Trait</code> to mean "always const":</p>
// This now compiles because `T` is always guaranteed to be `const`.
</span>const fn </span>foo</span><T: </span>const</span> Trait>() {
</span>    [(); <T as Trait>::func()];
</span>}
</span></code></pre>
This gets us into a difficult spot though. This would create a difference
between the meaning of const</code> when declared and when used in parameters.
Ideally we could, like, not</em> have that. Because as we've seen in other places
having the same word mean different things in different contexts can be pretty
confusing.</p>
The need to specialize</h2>
Another thing to mention here is the idea of specialization</em>. In the compiler
we have const_eval_select</code></a> which allows you to specialize depending
on whether you're executing at runtime or during compilation. This can be useful
to optimize implementations with, since more assumptions can be made about the
exact environment you'rer running on during runtime. During compilation Rust is
executed in an interpreter, which by design creates a uniform environment
regardless of what platform the compiler is actually running on.</p>
#![</span>feature</span>(const_eval_select)]
</span>#![</span>feature</span>(core_intrinsics)]
</span>use </span>std::intrinsics::const_eval_select;
</span>
</span>// A function which returns a different result based on runtime or compiletime
</span>pub const fn </span>inconsistent</span>() -> </span>i32 </span>{
</span>    </span>fn </span>runtime</span>() -> </span>i32 </span>{ </span>1 </span>}
</span>    </span>const fn </span>compiletime</span>() -> </span>i32 </span>{ </span>2 </span>}
</span>    </span>unsafe </span>{
</span>        </span>const_eval_select</span>((), compiletime, runtime)
</span>    }
</span>}
</span>
</span>const </span>CONST_X</span>: </span>i32 </span>= </span>inconsistent</span>();
</span>let</span> x = </span>inconsistent</span>();
</span>assert!(x != </span>CONST_X</span>);
</span></code></pre>
Something we're talking about in the initiative group is: "How can we enable
people to author different versions of the same resource which only
differentiate based on the effect?" This would effectively fill the same role as
const_eval_select</code>, but you could imagine it working for other effects such as
async</code> as well. The syntax we've been playing around is some version of:</p>
// maybe-async definition using an intrinsic to switch based on the context
</span>async<A> </span>fn </span>foo</span>() { </span>async_eval_select</span>(..) }
</span>
</span>// the equivalent overload-based variant
</span>async </span>fn </span>foo</span>() -> </span>usize </span>{ </span>1 </span>}   </span>// always async
</span>fn </span>foo</span>() -> </span>usize </span>{ </span>2 </span>}         </span>// never async
</span></code></pre>
Something we haven't really figured out is how we should define this for const</code>
definitions. We'd want to declare two variants: one which is definitely not
const. And one which definitely is. And have these create a unified definition.</p>
// maybe-const variant with the intrinsic
</span>const fn </span>foo</span>() -> </span>usize </span>{ </span>const_eval_select</span>(..) }
</span>
</span>// this wouldn't work because we can't define an "always const" context
</span>const fn </span>foo</span>() -> </span>usize </span>{}   </span>// sometimes const
</span>fn </span>foo</span>() -> </span>usize </span>{}         </span>// never const
</span></code></pre>
Summary: what is const?</h2>
Let's recap everything we've seen about const</code> so far. In today's Rust the const</code> effect
can roughly be thought of as having three different modes, split between
declaration and usage:</p>
</th> declaration</th> usage</th></tr></thead>

keyword never applies</strong></td> fn foo() {}</code></td> fn bar() { foo() }</code></td></tr>
keyword always applies</strong></td> -</td> const FOO: () = {};</code>   †</em></td></tr>
keyword conditionally applies</strong></td> const fn foo() {}</code></td> const fn bar() { foo() }</code></td></tr>
</tbody></table>

†: an "always const" context can only ever call "maybe-const" declarations
since there is no "always-const" declaration available in Rust today.</em></li>
</ul>
The const</code> keyword here means different things. const FOO</code> is used to declare
an "always-const" context. But when you write const fn foo</code> you're in fact
declaring a "sometimes-const" context. This means const</code> does not have a
consistent meaning in the language: depending on where it's used, it may either
signal an "always const" context, or a "maybe const" context.</strong> And as we've
seen in the case of trait bounds, there is a meaningful observable distinction
between "always const" and "maybe const" contexts, bounds, and declarations.</p>
This following table shows how "const definitions" interact with "const
contexts". A "never const" context is runtime-only, an "always const" context is
compilation-only, and a "sometimes const" context can be evaluated both during
compilation and at runtime:</p>
Can [def] be evaluated in [context]?</th> never const context</th> sometimes const context</th> always const context</th></tr></thead>

never const def</strong></td> ✅</td> ❌</td> ❌</td></tr>
sometimes const def</strong></td> ✅</td> ✅</td> ✅</td></tr>
always const def</strong></td> ❌</td> ❌</td> ✅</td></tr>
</tbody></table>
The way to think about "const" is as a subset of "base" Rust. We gain the
ability to evaluate code during compilation by removing capabilities from the
language. This means we don't get the ability to access statics, or any host
functionality. Because "const Rust" is a subset of "base Rust", you can also
think about it in an inverse and consider "base Rust" a superset of "const
Rust". If we pull in "async Rust" as well, we can plot the following diagram:</p>
                   +---------------------------+                               
</span>                   | +-----------------------+ |     Compute values:
</span>                   | | +-------------------+ | |     - types
</span>                   | | |                   | | |     - numbers
</span>                   | | |    const Rust     |-------{ - functions               
</span>                   | | |                   | | |     - control flow            
</span> Host access:      | | +-------------------+ | |     - traits (planned)                 
</span> - networking      | |                       | |     - containers (planned)
</span> - filesystem  }-----|      "base" Rust      | |                               
</span> - threads         | |                       | |                               
</span> - system time     | +-----------------------+ |     
</span> - statics         |                           |     Control over execution:      
</span>                   |        async Rust         |---{ - ad-hoc concurrency      
</span>                   |                           |     - ad-hoc cancellation/pausing/resumption
</span>                   +---------------------------+     - higher-order control flow; ad-hoc timeouts, etc.
</span></code></pre>
Back to the drawing board</h2>
Okay, analysis time stops here. What I've said so far is mostly factual, and I
think we should be able to reference as mostly true for the foreseeable future.
From this point onward we put our imagination caps on and strap in to speculate
about what could be</em>. If we could go back to the drawing board, how could we
change things?</p>
Well, for one, we now know we not only want to have "maybe const" functions and
types. We also want "maybe async", "maybe throws"; basically "maybe anything</em>"
effects. The fact that const fn</code> is "maybe" by default but async</code> isn't can be
a bit surprising. Here they are side-by-side:</p>
</th> keyword async</code></th> keyword const</code></th></tr></thead>

keyword never applies</strong></td> fn foo() {}</code></td> fn foo() {}</code></td></tr>
keyword always applies</strong></td> async fn foo() {}</code></td> const FOO: () = {};</code>†</em></td></tr>
keyword conditionally applies</strong></td> async<A> fn foo() {}</code> ‡</em></td> const fn foo() {}</code></td></tr>
</tbody></table>

†: "always-const" functions don't exist in Rust today, so an alternate method is shown here</em></li>
‡ : placeholder syntax for functions where the keyword conditionally applies</em></li>
</ul>
The difference is pretty striking right? We don't have an "always const"
function. And what we mean by "maybe const" in one place means "always async" in
the other. When we look at usage</em>, const</code> and async</code> look a lot more similar though:</p>
// Using `RFC 2920 experimental in-line const`
</span>let</span> value = </span>const </span>{ "</span>hello</span>" };
</span>
</span>// async block
</span>let</span> value = async { "</span>hello</span>" }.await;
</span></code></pre>
They're not quite</em> the same because they're not the same feature. But the usage
here feels pretty good already. Now how could we bring the rest of const</code> to
feel a bit more in line with the other keywords. Probably the boldest answer
would be to change the meaning of const fn</code> to mean: "always const". It could
then use a similar syntax to "maybe async" to declare "maybe const" contexts,
yielding us the following table:</p>
</th> keyword async</code></th> keyword const</code></th></tr></thead>

keyword never applies</strong></td> fn foo() {}</code></td> fn foo() {}</code></td></tr>
keyword always applies</strong></td> async fn foo() {}</code></td> const fn foo() {}</code> †</em></td></tr>
keyword conditionally applies</strong></td> async<A> fn foo() {}</code> ‡</em></td> const<C> fn foo() {}</code> ‡</em></td></tr>
</tbody></table>

† : "always-const" functions are new in this table, and use the existing "maybe-const" definition</em></li>
‡ : placeholder syntax for functions where the keyword conditionally applies</em></li>
</ul>
This would fix the main inconsistency between the effects, enabling them to be
used in a much more uniform manner. Looking at our earlier example, the original
formulation would have to become:</p>
// declraring a "maybe const" context which takes "maybe const" params
</span>const</span><C> </span>fn </span>foo</span><T: </span>const</span><C> Trait>() {
</span>    [(); <T as </span>const</span> Trait>::func()];
</span>    </span>//   ^ illegal; `T` must be "always const"
</span>}
</span></code></pre>
This, at least to me, feels like it more clearly describes the problem at hand.
const fn</code> and T: const</code> now use the same syntax to indicate they're
conditional. And we're trying to cast a T</code> which is "maybe const" to be "always
const". Diagnostics should make this fairly easy to point out.</p>
This means the solutions to this issue could also be a lot clearer: we either
cast T</code> to be "always const", or mark the const fn foo</code> to be an "always const
context":</p>
// maybe-const fn foo, with an always-const trait T
</span>const</span><C> </span>fn </span>foo</span><T: </span>const</span> Trait>() {
</span>    [(); <T as </span>const</span> Trait>::func()];
</span>}
</span>
</span>// always-const fn foo, with an always-const trait T
</span>const fn </span>foo</span><T: Trait>() {
</span>    [(); <T as Trait>::func()];
</span>}
</span></code></pre>
I'm less sure about having const fn foo</code> imply that all generics are also
const</code>, all casts are const</code>, etc. Maybe they should be specified as const</code>
as well? But on the other hand: they would have to be const</code> for it to work, so
maybe it can be ommitted? Though that's mostly a syntactical question, as it
wouldn't affect the semantics.</p>
Making the meaning of const</code> unambiguous seems like it would make cases like
these a lot easier to work with. And it doesn't just stop at bounds; if we
wanted to enable effect-based overloading, the example we saw earlier would Just
Work:</p>
// maybe-const variant with the intrinsic
</span>const</span><C> </span>fn </span>foo</span>() -> </span>usize </span>{ </span>const_eval_select</span>(..) }
</span>
</span>// under the new rules this would be equivalent to a single "maybe const"
</span>// declaration:
</span>const fn </span>foo</span>() -> </span>usize </span>{}   </span>// always const
</span>fn </span>foo</span>() -> </span>usize </span>{}         </span>// never const
</span></code></pre>
If we were to introduce this change, I believe we could actually do it over an
edition bound. The main "new" feature is the ability to declare "always-const"
functions, traits, methods, etc. These couldn't be declared in older editions;
but because changing an existing "maybe-const" definition to become
"always-const" would be considered a breaking change, it means that it wouldn't
just break existing code simply by existing.</p>
const fn</code> code written on an older edition would be reinterpreted as
"maybe-const" in newer editions. And possibly we could also just add support for
trait bounds through const<C></code> on the old edition too, so on older editions
there would just be multiple ways to declare similar "maybe const" contexts:</p>
//! The different meanings of `const` between editions
</span>
</span>// old edition
</span>const fn </span>foo</span>() {}                     </span>// maybe-const no traits
</span>const</span><C> </span>fn </span>foo</span><T: </span>const</span><C> Foo>() {} </span>// maybe-const with traits
</span>
</span>// new edition
</span>const fn </span>foo</span>() {}                     </span>// always const
</span>const</span><C> </span>fn </span>foo</span>() {}                  </span>// maybe-const no traits
</span>const</span><C> </span>fn </span>foo</span><T: </span>const</span><C> Foo>() {} </span>// maybe-const with traits
</span></code></pre>
//! Even if older editions can't define always-const functions,
</span>//! they could still be available to older editions. The hardest part
</span>//! is probably figuring out the diagnostics for the older edition to talk
</span>//! about a concept which can be used, but not created.
</span>
</span>// new edition: defines an always-const function
</span>crate</span> new {
</span>    </span>const fn </span>foo</span>() {}    </span>// always const declaration
</span>}
</span>
</span>// old edition; uses an always-const function
</span>crate</span> old {
</span>    </span>use super</span>::new::foo;
</span>    </span>const</span> Foo: () = </span>foo</span>();  </span>// okay to call an always-const fn in an old edition
</span>                            </span>// but no way to declare new ones
</span>}
</span></code></pre>
As I've mentioned before: the exact syntax for conditional effects is still
undecided. But regardless of what we end up with, the idea is that we can create
something consistent between keywords.</p>
Conclusion</h2>
In this post we've gone over what const</code> is, how it relates to other keywords,
and gone deeper on the different uses and meanings of const</code> in Rust today.
We've then taken a shot at trying to resolve these issues, with an outline of
how we could potentially normalize the meaning of the const</code> keyword over an
edition bound.</p>
Thanks to Oli</a> for proof reading this post!</em></p>

Inline Crates
Tue, 25 Oct 2022 00:00:00 +0000
People sometimes jest that Rust is just ML dressed up to look like C++. And I
don't think that's entirely off: Rust has many of the key features present in ML
languages. We have the same kind of type system (Hindley-Milner</a>), we have
sum types, and we have a module system which isn't directly tied to a module
hierarchy. I want to talk a bit more about Rust's module system here.</p>
In Rust we distinguish between "crates"</a> and "modules"</a>. To people just
learning about Rust the distinction can be a bit confusing. But in practice it
makes sense to have both. In this post we're going to take a look at Rust's
module system, what the differences are, and how we could introduce some
features to bring crates and modules closer together.</p>
Disclaimer: In this post I'm sharing some ideas on language design that heavily tie into
the current compiler architecture. I want to make it super clear that I'm not
necessarily advocating we make these changes, and I'm especially not trying to
say that this should be prioritized. I just wanted to have this written down so
it can be referenced later. Because I think there's value in writing things
down even if they're not fully formed, feasible, or quite the right time.</em></p>
What are crates?</h2>
A crate is a unit of code with strict</em> encapsulation rules: Crates are also
a unit which can individually be published to and pulled from crates.io. In the
compiler it represents a translation unit, and is the boundary of parallelization 1</a></sup>.
We're only able to compile entire crates in parallel; not yet sub-components of
crates. And it also affects the boundaries of re-compilations.</p>
^1</sup>This is only about rustc</code> specifically - the LLVM part of the
compilation may parallelize individual crates. You can read more on this in the
rustc dev guide</a> and the rustc book</a>.</p>
</div>
Crates themselves also have strict rules on disallowing cyclic dependencies
2</a></sup>, and of course the orphan rules</a> 3</a></sup>. A crate is always
tied to a file hierarchy on disk, and needs an accompanying Cargo.toml</code> file.
Matklad (of Rust-Analyzer fame) goes into detail on their blog</a> how crates
affect compilation times, and how changing crate hierarchies can be used to
significantly improve throughput.</p>
^{2</sup>
Given a</code> and b</code>. a</code> can depend on b</code>, or b</code> can depend on a</code>.
But they can't both depend on each other, because that would cause a cycle in the dependency graph.</p>
</div>}^{3</sup>
Orphan rules come down to: if you want to implement a trait for
a type, you must either define the trait or the type in your crate. You're not
allowed to implement a trait you haven't defined for a type you haven't defined.</p>
</div>}Finally, some features such as procedural macros</a> can only be defined in crates
which have a specific configuration that disallows any other type of code to be
exported. Crates, more generally, also serve as a boundary of configuration</em>.</p>
backyard
</span>├── Cargo.lock
</span>├── Cargo.toml
</span>└── src
</span>    ├── garden
</span>    │   └── vegetables.rs
</span>    ├── garden.rs
</span>    └── lib.rs
</span></code></pre>
An example of a crate representation, copied from the Rust book</a>.</em></p>
For the purpose of this post we'll not be talking on the Rust 2015-era concept
of extern crate</code></a>.</p>
What are modules?</h2>
A module is a unit of code with loose</em> encapsulation rules: Cycles between
modules are allowed, and a module does not necessarily correspond to a file
hierarchy. Crates are a lightweight way of splitting code up into logical
components. Here's an example of a valid mod</code> relationship
(playground</a>):</p>
// example of cycles in modules
</span>
</span>pub mod </span>foo {
</span>    </span>use super</span>::bar::*;
</span>    </span>pub struct </span>First;
</span>}
</span>
</span>pub mod </span>bar {
</span>    </span>use super</span>::foo::*;
</span>    </span>pub struct </span>Second;
</span>}
</span></code></pre>
But modules can also correspond to a file hierarchy. In the following example,
backyard</code> is the crate, with lib.rs</code> representing the entry point, and
internally contains crate::garden</code>, and crate::garden::vegetables</code> as
sub-modules:</p>
backyard
</span>├── Cargo.lock
</span>├── Cargo.toml
</span>└── src
</span>    ├── garden
</span>    │   └── vegetables.rs
</span>    ├── garden.rs
</span>    └── lib.rs
</span></code></pre>
Modules also accept visibility modifiers such as pub(crate)</code> or pub(super)</code>. This
restricts the encapsulation rules somewhat by making the code accessible from
fewer sites. It also changes things like being able to access fields on structs.
But while it's stricter</em>, it still allows cycles between modules. Which is not
the same level of guarantees which crates provide.</p>
comparing modules and crates</h2>
To summarize what we've covered so far:</p>
name</th> encapsulation</th> file repr</th> in-source repr</th> cost to define and maintain</th></tr></thead>

crate</code></td> strict</td> ✅</td> ❌</td> high</td></tr>
mod</code></td> loose</td> ✅</td> ✅</td> low</td></tr>
</tbody></table>
Crates and modules are both incredibly useful concepts, with benefits and
tradeoffs. But today it's significantly harder to define new crates, than it is
to define new modules. Features like "workspace inheritence"</a> lower the barrier
somewhat, but there's still a pretty large difference between how easy they are
to introduce to your project.</p>
Adding a new module is basically just adding a mod {}</code> and copying over the
right imports - something which Rust-Analyzer can even do for you. But adding a
new crate is far more involved: it requires creating a new module hierarchy,
adding the right dependencies, adding the right imports to the workspace, and
then linking the right imports back to the crates you want to use. This could be
automated as well, but that only makes authoring crates easier - it doesn't help
with readability.</p>
And that's not even taking into account releasing</em> crates. When you add a new
mod</code> to a crate it doesn't affect the ability to publish new crates. But when
you create a new crate, you now have a new dependency. Which must be accurately
versioned 4</a></sup> and published before the main crate can be released. The
fact that "strict encapsulation semantics" are necessarily tied to an on-disk
hierarchy is not ideal.</p>
^{4</sup>
"workspace inheritence" can also help with versioning here.</p>
</div>}in-line modules</h2>
You can probably see where this is going. In my opinion Rust would benefit from
detaching crates from the module hierarchy, and allowing them to also be defined
in a manner similar to modules. A single "crate" should be able to contain many
sub-crates, all of which enforce the same strict encapsulation rules as their
on-disk counterparts. The only difference being in how they're declared. Much
like workspace.inherit</code>, sub-crates would inherit the dependencies and version
number of their parent crate. And unlike on-disk crates, they wouldn't be
individually publishable to crates.io, but instead just be sent along with their
parent crates.</p>
To give an example of what this would look like; I'm thinking we adopt a very
similar syntax to modules:</p>
mod </span>foo {}   </span>// define an in-line module
</span>crate</span> foo {} </span>// define an in-line crate
</span>
</span>mod </span>bin;     </span>// import a module from a file
</span>crate</span> bin;   </span>// import a crate from a file
</span></code></pre>
Example</h2>
Being able to distinguish at the source-level between "public" and "private"
crates is something which would be useful for larger projects in particular. An
example of such a project is probably Rust-Analyzer, which I happen to be
familiar with.</p>
You can see an overview of all crates in the code map</a> section of the
architecture.md</a> file. Rust-Analyzer clearly distinguishes between "API
boundary"</a> crates, and all other crates. For example something like the ide</code>
crate is a "boundary", but internally uses other crates such as ide-db</code>, and
ide-assists</code> which are not boundaries:</p>
crates/
</span>    </span>hir-def/
</span>    </span>hir-expand/
</span>    </span>hir-ty/
</span>    </span>hir/             </span># boundary
</span>    </span>ide-assists/
</span>    </span>ide-completion/
</span>    </span>ide-db/
</span>    </span>ide-diagnostics/
</span>    </span>ide-ssr/
</span>    </span>ide/             </span># boundary
</span></code></pre>
I've contributed to Rust-Analyzer in the past, but I'm not on the RA team - so I
can't speak with any authority about the project. But it's not unreasonable that
a project similar to Rust-Analyzer could benefit from only surfacing the API
boundary crates in the top-level crate hierarchy - without needing to give up
the strict encapsulation rules Rust crates provide.</p>
crates/
</span>    </span>hir/             </span># boundary
</span>        </span>def/
</span>        </span>expand/
</span>        </span>ty/
</span>    </span>ide/             </span># boundary
</span>        </span>assists/
</span>        </span>completion/
</span>        </span>db/
</span>        </span>diagnostics/
</span>        </span>ssr/
</span></code></pre>
There may be projects who prefer to use a flat module hierarchy even if in-line
crates become available. But when starting it seems easier to be able to quickly
define a new crate, and only later move it to its own hierarchy. The point of
the crate</code> keyword is to enable strict encapsulation without immediately having
to resort to separate on-disk representations. Which should make it possible to
let the structure found during prototyping hold all the way. As opposed of the
current status quo where prototypes can be started using mod</code> statements, but
eventually you'll want to refactor into separate crates.</p>
Implementation</h2>
I suspect implementing inline crates might not be an easy task. If I'm not
mistaken 5</a></sup>, right now parallelism of compilation is driven by
cargo</code> spawning a bunch of instances of rustc</code> which compile individual
crates. If crates are represented as anything other than something you can point
rustc(1)</code> to compile, it might need some rethinking of the architecture.</p>
I don't think that's an argument against</em> inline crates though. But rather a
reflection that the architecture of how we've organized the various rust
compiler projects is reflected in the features provided it provides 6</a></sup>. And things
features such as inline exist right at the seems of where the two projects meet.</p>
^{5</sup>
Wesley checked this and said my understanding of this is indeed
correct. I really wasn't sure about this in earlier drafts, but I'm thankful I
was able to confirm ^^.</p>
</div>}^6</sup>See: Conway's law</a>.</p>
</div>
Future Directions</h2>
pub crate</h3>
Alright, time to speculate a little. Conversations around "crates.io namespaces"
pop up pretty regularly. The basic idea is that sometimes crates are
correlated, and being able to group them together under a single namespace would
be nice.</p>
Take for example the windows</code> crate</a>. It's probably the single biggest crate
on crates.io 7</a></sup>. Internally it's built up of lots of crates which are all
versioned in lock-step, and hidden behind feature flags.  Take for example
win32</code>'s HTTP service API. To use it with the windows</code> crate you'd need to
define it in your toml like so:</p>
^7</sup>I believe it exposes something like 30.000 unique types and traits.
Like an order of magnitude more than e.g.
web-sys</code></a> does, which covers the
entire web platform</em>. windows</code> is a pretty good example for a "big" crate that
needs to be broken up into sub-components for the compiler to even attempt to
build it in reasonable time.</p>
</div>
[dependencies]
</span>"</span>windows</span>" = { </span>version </span>= "</span>0.41.0</span>", </span>features </span>= ["</span>Win32_Networking_HttpServer</span>"] }
</span></code></pre>
Instead, if the windows</code> crate could be used as a namespace, we could imagine
that instead of exclusively exposing APIs using feature flags, we may want to
expose it as crates namespaced under windows</code>. For example, we could imagine
something like this:</p>
[dependencies]
</span>"</span>windows/win32_networking_httpserver</span>" = "</span>0.41.0</span>"
</span></code></pre>
There are some big "ifs" involved here: we're assuming this can be neatly split
out into a single crate. But let's assume we can. How would we split it out?
Today that would mean creating a new hierarchy on disk, and putting the right
code there. And then in turn resolving the dependency hierarchy ahead of time to
correctly version and publish the dependencies in the right order.</p>
What if we could have an 1:N</code> mapping to published crates from within crates?
What if the definition inside the windows crate could look like this and it
would be enough to both provide the feature from inside the windows</code> crate, and
be publicly accessible as a sub-crate?:</p>
#[</span>cfg</span>(feature = "</span>Win32_Networking_HttpServer</span>")]
</span>pub crate</span> win32_networking_httpserver { ... }
</span></code></pre>
I'd imagine the dependencies and versions would all be shared from the
Cargo.toml</code> definition. And dependencies between inline crates would be
extrapolated and extracted into the manifest. I don't think this should
replace</em> workspaces in any way. Workspaces are a great feature, and give a
great deal of control. But it seems like it could be a way to lower the overall
burden of authoring new packages.</p>
WASI components</h3>
And to continue the speculation: I've been wondering for a few months now
whether we could provide an 1:N</code> mapping of Rust programs to WASI
components</a>. In WebAssembly
components serve as a boundary of strict encapsulation, not unlike but als not
exactly along the lines of crates in Rust.</p>
I'm fairly optimistic that WASI will present a best-in-class target to compile
Rust to for networked services. And if we're able to provide fine-grained and
easy-to-use mappings from Rust to WASI components, it might give Rust an edge
over other languages. I suspect being able to lean into the strict encapsulation
semantics of Rust crates may have meaningful benefits wrt compile targets that
we're currently unable to leverage to its full potential 8</a></sup>.</p>
^{8</sup>
I'm trying not to go full in on WASI encapsulation rules, and mapping
Rust programs to WASI. But I think there's a lot there, and being able to define
in-line crates may end up being a component in a full WASI story.</p>
</div>}Conclusion</h2>
That's about it I think. I would love to see more equivalence between "crates"
and "modules" in Rust. Even if we one day see a loosening of the orphan rules,
or we see Rust move to fine-grained paralellism - having in-line crates will
still be useful, as they're easier to create and modify than their on-disk
counterparts. Even if it's just to prototype and experiment before wanting to
commit to a final design.</p>
To come entirely clean: I don't see this work panning out anytime soon. It would
require an effective rearchitecture of the compiler, and by itself this feature
wouldn't carry its weight to justify doing that. But perhaps if enough reasons
build up, eventually we'll look to make the changes which would also</em> enable
this feature. Either way, I figured it'd be worth spelling this out.</p>
I think we could ask some very interesting questions about what other benefits
such a rearchitecture of the compiler could look like as well. Could compilation
become more efficient? Could it become easier to optimize builds? What would we
lose? What other features could be enabled? Even if we know we don't have the
time for any of this now, I do think it's 1: fun to think about, and 2: perhaps
something useful will come out of it.</p>
I also want to at least state once that I don't believe this should be in place
of any workspace improvements. But I see working in addition to</em> workspace
improvements. I view both approaches as improvements to status quo, and I think they would end up complimenting each other rather well!</p>
Thanks to Wesley Wiser</a> for proof reading!</em></p>

Why Async Rust
Mon, 26 Sep 2022 00:00:00 +0000
A lot of system design is about thinking of the nature of the domains we
encounter. And only later, once we understand them, encoding this understanding
in a way that machines can verify it.</p>
I often find async Rust to be misunderstood. Conversations around "why
async" often focus on performance 1</a></sup> - a topic which is highly dependent on
workloads, and results with people wholly talking past each other. While
performance is not a bad</em> reason to choose async Rust, we often we only notice
performance when we experience a lack of it. So I want to instead on which
features</em> async Rust provides which aren't present in non-async Rust. Though
we'll talk a bit about performance too at the end of this post.</p>
^1</sup>The introduction to the Rust async book</a> summarizes the
benefits of async Rust as follows: "In summary, asynchronous programming allows
highly performant implementations that are suitable for low-level languages like
Rust, while providing most of the ergonomic benefits of threads and coroutines."</p>
</div>
Language Hierarchy</h2>
It's not uncommon to hear Rust and other languages described as "N languages in
a trenchcoat". In Rust we have Rust's control flow constructs, we have the
decl-macro meta language, we we have the trait system (which is
turing-complete</a>), we have the cfg annotation language - and the list keeps
going. But if we consider Rust as it is provided out of the box to us, as "base
Rust" then there are some obvious modifiers to it:</p>

unsafe</code> Rust: to use raw pointers and FFI</li>
const</code> Rust: to compute values at compile-time</li>
async</code> Rust: to enable non-blocking computation</li>
</ul>
All of these "modifier keyword" to the Rust language provide new capabilities
which aren't present in "base Rust". But they may sometimes also take
capabilities away. The way I've started thinking and talking about language
features is in terms of "subset of the language" or "superset of the language".
With that classification we can look at the modifier keywords again and make the
following categorization:</p>

unsafe</code> Rust: superset</li>
const</code> Rust: subset</li>
async</code> Rust: superset</li>
</ul>
unsafe</code> Rust only adds</em> the ability to use raw pointers. async</code> only adds the
ability to .await</code> values. But const</code> adds</em> the ability to compute values
during compilation, but removes</em> the ability to use statics and access things
like the network or filesystem.</p>
If language features only add</em> to base Rust then they're considered supersets.
But if the features they add require they also restrict other features, then
they're considered subsets. In the case of const</code>, all const</code> functions can be
executed at runtime. But not all code which can be executed at runtime can be
marked as const</code>.</p>
The design of language features as sub/supersets of "base" Rust is essential: it
ensures that the language keeps feeling cohesive</em>. And more so than size or
scope, uniformity is what leads to the feeling of simplicity</a>.</p>
async under the hood</h2>
At its core async/.await</code> in Rust provides a standardized way to return types
with a method on them that returns another type. Instead of returning a type
directly, async</code> functions return an intermediate type first. 2</a></sup></p>
^{2</sup>
Sure; "monad" 🙃</p>
</div>}/// This function returns a string
</span>fn </span>read_to_string</span>(</span>path</span>: Path) -> String { .. }
</span>
</span>/// This function returns a type which eventually returns a string
</span>async </span>fn </span>read_to_string</span>(</span>path</span>: Path) -> String { .. }
</span>
</span>/// Instead of using `async fn` we can also write it like this.
</span>/// `impl Future` here is a type-erased struct
</span>fn </span>read_to_string</span>(</span>path</span>: Path) -> impl Future<Output = String> { .. }
</span></code></pre>
A future is just a type with a method on it (fn poll</code></a>). If we call the method
at the right times and in the right way, then eventually it'll give us the
equivalent of Option<T></code> where T</code> is the value that we wanted.</p>
"A future is just a type with a method we can call" has a few implications.
First of all, just like we can choose to call the method, we can also choose to
not call the method. If we don't call the method at all then the future won't
execute any work 3</a></sup>. Or we can choose to call it, and then stop</em>
calling it for a while. Maybe we just call the method again later. We can even
choose to drop the struct at any point, and then there's no more method to call
and no more value to be obtained.</p>
^{3</sup>
Yes, that's only the case for async fn / async {}</code>-futures. If
you -> impl Future</code> you can do work before</em> constructing the future and
returning it. But that's not considered a great pattern, and it's pretty rare
in practice.</p>
</div>}When we talk about "representing a computation in a type", we're actually
talking about compiling down the async fn</code> and all of its .await</code> points into
a state machine which knows how to suspend and resume from the various .await</code>
points. These state machines are just structs with some fields in them, and
and have an auto-generated Future::poll</code> implementation which knows how to
correctly transition between the various states. To learn more about how
these state machines work, I recommend watching "life of an async fn" by
tmandry</a>.</p>
The .await</code> syntax provides a way to ensure that none of the underlying poll</code>
details surface in the user-syntax. Most usage of async/.await</code> looks just like
non-async Rust, but with async/.await</code> annotations sprinkled on top.</p>
Rust's Async Features</h2>
The core feature async/.await</code> provides in Rust is control over execution</em>.
Instead of the contract being:</p>
> "function call" -> "output"
</span></code></pre>
We instead get access to an intermediate step:</p>
> "function call" -> "computation" -> "output"
</span></code></pre>
The computation isn't just something which is hidden away from us anymore. With
async/.await</code> we are empowered to manipulate the computation itself. This leads
to several key capabilities:</p>

The ability to suspend/cancel/pause/resume computation (ad-hoc cancellation)</li>
The ability to execute computations concurrently (ad-hoc concurrency)</li>
The ability to combine control over execution, cancellation, and concurrency</li>
</ul>
ad-hoc cancellation</h3>
The ability to suspend/cancel/pause/resume any computation is incredibly useful. Out of the
three being able to cancel execution is likely the most useful one. In
both sync and async code it's both desireable to halt execution before it
completes. But what's unique to async Rust is that any</em> computation can be
halted in a uniform way. Every future can be cancelled, and all futures need to
account for that 4</a></sup>.</strong></p>
^4</sup>Yes I'm fully aware of the concept of "cancellation-safety",
and I have a post coming up discussing it in more detail. The tldr:
"cancellation-safety" as a concept is under-specified, but importantly:
"cancellation-safety" is only relevevant when using select! {}</code>, which is
something which should not be
used</a>. Correctly
handling cancellation is something that manually authored futures still need to
do though, and that can be tricky to do without "async Drop". But that's
different from "cancellation-safety" or the idea that futures don't need to have
to account for being cancelled at all.</p>
</div>
ad-hoc concurrency</h3>
The ability to execute computations concurrently is another hallmark capability
of async Rust. Any number of async fn</code>s can be run concurrently 5</a></sup>,
and .await</code>ed together. In non-async Rust concurrency is typically tied
to parallelism: many computations can be scheduled concurrently by using
thread::spawn</code> and splitting it up that way. But async Rust separates concurrency
from parallelism, providing more control 6</a></sup>. In non-async Rust concurrency
and parallelism are interlinked, which among other things has performance
implications. We'll talk more about the differences later in this post.</p>
^{5</sup>
Bar any runtime faults such as deadlocks, which can occur if two
computations are run concurrently but share a resource.</p>
</div>}^{6</sup>
Not all libraries make use of the separation between "concurrency" and
"parallelism" though. We're still very much figuring out what async Rust even
is, but many of the libraries in common use today don't necessarily surface this
insight. I know many of my own older libraries sure don't.</p>
</div>}combining cancellation and concurrency</h3>
Now finally: when happens when you combine cancellation</em> and concurrency</em>? It
allows us to do some interesting things! In my blog post "Async Time III:
Cancellation and Signals"</a> I
go in-depth on some of the things you can do with this. But the canonical
example here is: timeouts. A timeout is a concurrent execution of some future
and a timer future, mapped to a Result</code>:</p>

If the future completes before the timer, we cancel the timer and return Ok</code></li>
If the timer completes before the future, we cancel the future and return Err</code></li>
</ul>
That's cancellation + concurrency combined to provide a new third type of
operation. To get a sense for why being able to time-out any computation is a
useful property, I highly recommend reading Crash-Only Software by Candea and
Fox</a> 7</a></sup>. But it doesn't just stop at timeouts: if we combine any of the
suspend/cancel/pause/resume capabilities with concurrency, we unlock a myriad of
new possible operations.</p>
^{7</sup>
Shout out to Eric Holk for introducing me to this paper!</p>
</div>

These are the features async Rust enables. In non-async Rust concurrency,
cancellation, and suspensions often require calling out to the underlying operating
system - and it's not always supported. For example: Rust doesn't have a
built-in way to cancel threads. The way to do it is usually to pass a channel to
a thread, and periodically check it to see if some "cancel" message has been
passed.</p>
In contrast in async Rust any</em> computation can be paused, cancelled, or run
concurrently. That doesn't mean that all computations should be run
concurrently, or everything should have a timeout on it. But those decisions can
be made on the basis of what we're implementing, rather than being limited by
external factors such as system call availability.</p>}Performance: workloads</h2>
When something is stated to perform better than something else, it's always
worth asking: "under which circumstances?"</em> Performance is  always dependent on
the workload. In graphics cards benchmarks you'll often see differences between
video cards based on which games are run. In CPU benchmarks it matters a lot
whether a workload is primarily single-threaded or multi-threaded. And when we
talk about software features "performance" is not a binary either, but highly
dependent on the workload. When talking about parallel processing we can
distinguish between two general categories of workloads:</p>

throughput-oriented workloads</li>
latency-oriented workloads</li>
</ul>
Thoughput-oriented workloads usually care about processing the maximum number of
things in the shortest amount of time. While latency-oriented workloads care about
processing each thing as quickly as possible. Sounds confusing? Let's make it
more clear.</p>
An example of software designed with throughput in mind is
hadoop</a>. It's built for
"offline" batch processing of workloads; where the most important design goal is
to minimize the total CPU time spent to process data. When data is put into the
system it may often take minutes or even hours to be processed. And that's fine.
We don't care when</em> we get the results (within reason of course), we primarily
care about using as few resources as possible to get the results.</p>
Compare that to a public-facing HTTP server. Networking is typically
latency-oriented</em>. We often care less about how many requests we can handle,
than how quickly we can respond to them</em>. When a request comes in we don't want
to take minutes or hours to generate a response. We want request-response
roundtrips to be measured in at most milliseconds. And things like
p99 tail-latencies</a> are often used as key performance indicators.</p>
Async Rust is generally considered to be more latency-oriented than
throughput-oriented. Runtimes such as async-std</code> and tokio</code> primarily care
about keeping overall latency low, and preventing sudden latency-spikes.</p>
Understanding what type of workload is being discussed is often the first step
in discussing performance. A key benefit of async Rust is that most of the systems
which use it have been tuned heavily to provide good performance for
latency-oriented workloads - of which networking is an example. If you want
to handle more throughput-oriented workloads, non-async crates like
rayon</code></a> are often a better fit.</p>
Performance: optimizations</h2>
Async Rust separates concurrency</em> and parallelism</em> from each other. Sometimes
the two are confused with each other, but they are in fact different:</p>

parallelism is a resource, concurrency is a way of scheduling computation</p>
</blockquote>
It's best to think of "parallelism" as a maximum. For example, if your computer has two cores, the maximum amount of parallelism you have may be two 8</a></sup>.
But parallelism is different from concurrency: computation can be interleaved on a single
core, so while we're waiting on the network to perform work, we can run some other
computations until we have a response. Even on single-threaded machines,
computation can be interleaved and concurrent. And vice-versa: just because we've
scheduled things in parallel doesn't mean computation is interleaved. No matter
how many cores we're running on, if threads are taking turns by waiting on a
single, shared lock, then the logical execution may in fact still happen sequentially.</p>
^8</sup>This is a simplified example; for a longer explainer see the
std::thread::available_parallelism</a>
docs.</p>
</div>
Let's take a look at an example of a concurrent workload in async and non-async
Rust. In non-async Rust it's most common to use threads to achieve concurrent
execution 9</a></sup>. But since threads are also the abstraction to achieve
parallel</em> execution, it means that in non-async Rust concurrency</em> and
parallelism</em> are often tightly interlinked.</p>
In async Rust, we can separate concurrency from parallelism. If a workload is
concurrent</em>, it doesn't imply it is also parallelizable</em> as well. This
provides finer control over execution, which is the key strength of async Rust.
Let's compare concurrency in non-async and async Rust:</p>
^9</sup>Unless you start manually writing
epoll(7)</code></a> loops, and hand-roll state
machines. At some point you may start thinking about creating abstractions to
make these state machines compose better with each other, at which point you've
basically arrived at futures again. Natively, in order to achieve: "I want to
make this code run concurrently", threads are the easiest, most convenient
abstraction available in non-async Rust.</p>
</div>
// thread-based concurrent computation
</span>let</span> x = thread::spawn(|| </span>1 </span>+ </span>1</span>);
</span>let</span> y = thread::spawn(|| </span>2 </span>+ </span>2</span>);
</span>let </span>(x, y) = (x.</span>join</span>(), y.</span>join</span>()); </span>// wait for both threads to return
</span>
</span>// async-based concurrent computation
</span>let</span> x = async { </span>1 </span>+ </span>1 </span>};
</span>let</span> y = async { </span>2 </span>+ </span>2 </span>};
</span>let </span>(x, y) = (x, y).await;  </span>// resolve both futures concurrently
</span></code></pre>
This may seem like a pretty silly example: computation is synchronous and so
both do the same thing, but the non-async variant has the overhead of needing to spawn actual
threads. And it doesn't stop there: because the second example doesn't need
threads, the compiler's inliner can kick in, and may be able to optimize it to
the following 10</a></sup>:</p>
// compiler-optimized async-based concurrent computation
</span>let </span>(x, y) = (</span>2</span>, </span>4</span>);
</span></code></pre>
In contrast, the best optimization the compiler can likely perform for the
thread-based variant is:</p>
// thread-based concurrent computation
</span>let</span> x = thread::spawn(|| </span>2</span>);
</span>let</span> y = thread::spawn(|| </span>4</span>);
</span>let </span>(x, y) = (x.</span>join</span>(), y.</span>join</span>()); </span>// wait for both threads to return
</span></code></pre>
^10</sup>Note that this exact example doesn't yet work in the optimizer, but
that I don't believe there is any strong reason why it couldn't either. It's all
local reasoning, with no cross-thread synchronization needed. The closest
example I have for optimizations of this kind is this example of a hand-rolled
version of block_on</code></a> which compiles down to
absolutely nothing. This cheats a little bit by removing not using an
atomic-based Arc</code>, so I'm not sure how realistic it is. But it's def something
to aspire to, and I'm optimistic that as async Rust sees more usage, we'll see
more async-specific optimizations as well.</p>
</div>
Separating concurrency from parallelism allows for more optimizations of
computations. async</code> in Rust is basically a fancy way of authoring a state
machine, and nested async/.await</code> calls allow types to be compiled down into
singular state machines. Sometimes we may want to separate the state
machines</a> though, but
that's the sort of control which async Rust provides us with, which is harder to
achieve using non-async Rust.</p>
Ecosystem</h2>
Before we close out, we should point out one last
reason people may choose async Rust: the size of the ecosystem. Without keyword
generics</a> it can be a lot of work for library authors to publish and maintain
libraries which work both in async and non-async Rust. Often it's easiest to
just publish either an async or non-async library, and not account for the other
use case. A lot of the network-related libraries on crates.io use async Rust
though, which means that libraries building on top of this will also use async
Rust. And in turn people looking to build websites without rewriting everything
from scratch will often have a larger ecosystem to choose from when using async
Rust.</p>
Network effects are real and need to be acknowledged in this context. Not everyone wanting to
build a website will be thinking in terms of language features, but may instead
just be looking at the options they have in terms of ecosystem. And that's a
perfectly valid reason to use async Rust for too.</p>
Conclusion</h2>
Sometimes conversations pop up about async Rust with suggestions such as:
"What if we disallowed cancellation in its entirety?" 11</a></sup> Seeing this
always confuses me, because it seems it carries a fundamental misunderstanding
of what async Rust provides and why it should be used. If the focus is solely
on performance, features such as cancellation or .await</code> annotations may seem
like a plain nuisance.</p>
^{11</sup>
The goal of this is often to have linear-futures, or "futures
which are guaranteed to complete". The way by which cancellation would be
disallowed is only the "stop polling" kind. You should still be able to pass
channels around to cancel things, at least that's the theory. Though I believe
there may also be implications for concurrency, which would make this really
tough.</p>
</div>
But if the focus is more on the features async Rust enables, things like
cancellation and timeouts quickly rise from nuisance to key reasons to adopt
async Rust. Async Rust grants us the ability to control execution in a way which
just isn't possible in non-async Rust. And frankly, isn't even possible in many
other programming languages featuring async/.await</code> either. The fact that an
async fn</code> compiles down to a lazy state machine instead of an eager managed
task is a crucial distinction. And it means that we can author concurrency
primitives entirely in library code, rather than needing to build it into the
compiler or runtime.</p>
In async Rust the features it enables build on each other. Here's a brief
summary of how they relate:</p>}            Yosh's Hierarchy
</span>        of Rust Async Capabilities
</span>    ┌───────────────────────────────┐
</span>3.  │   Timeouts, Timers, Signals   │  …which can then be composed into…
</span>    ├───────────────┬───────────────┤
</span>2.  │ Cancellation  │  Concurrency  │  …which in turn enable…
</span>    ├───────────────┴───────────────┤
</span>1.  │    Control over Execution     │  The core futures enable…
</span>    └───────────────────────────────┘
</span></code></pre>
We also briefly covered the performance aspect of async Rust. Generally speaking
it can</em> be more performant than non-async Rust when you're doing async IO. But
that will mostly be the case when the underlying system APIs are geared for
that, which usually includes networking APIs, and more recently has also started
including disk IO.</p>
Thanks to Iryna Shestak</a> for proof reading this post and providing helpful feedback along the way.</em></p>

Futures Concurrency IV: Join Ergonomics
Mon, 19 Sep 2022 00:00:00 +0000
On Thursday this week Rust 1.64 will be released, and in it it will include a
stabilized version of IntoFuture</code></a>. Much like IntoIterator</code> is used in the
desugaring of for..in</code> loops, IntoFuture</code> will be used in the desugaring of
.await</code>.</p>
In this post I want to show some of the ergonomics improvements IntoFuture</code> might
enable, inspired by Swift's recent improvements in async/await</code> ergonomics.</p>
async let in swift</h2>
Swift has recently added support for async let</code></a> 1</a></sup> in the language.
This provides lightweight syntax to create Swift tasks, which before this
proposal always had to be constructed explicitly within a task group</em>. In the
evolution proposal they show an example like this:</p>
^1</sup>J. McCall, J. Groff, D. Gregor, and K. Malawski, “SE-0317: async let bindings,” Mar. 25, 2021</a>. (accessed Apr. 07, 2022).</p>
</div>
func </span>makeDinner() async -> Meal {
</span>    </span>// Create a task group to scope the lifetime of our three child tasks
</span>    </span>return try</span> await withThrowingTaskGroup(of: CookingTask.</span>self</span>) { group </span>in
</span>    </span>// spawn three cooking tasks and execute them in parallel:
</span>    group.async {
</span>        CookingTask.veggies(</span>try</span> await chopVegetables())
</span>    }
</span>    group.async {
</span>        CookingTask.meat(await marinateMeat())
</span>    }
</span>    group.async {
</span>        CookingTask.oven(await preheatOven(temperature: </span>350</span>))
</span>    }
</span>    </span>// ... a whole lot more code after this
</span>}
</span></code></pre>
That's quite a bit of code. And async let</code> exists to simplify that. Instead of
explicitly creating the task group and manually creating tasks within it, using
async let</code> the right scope is inferred for us 2</a></sup>, and instead the
concurrency is expressed at the await</code> point later on:</p>
^{2</sup>
I may be wrong on the details here. It's been a sec since I last read
the Swift post, and I only glanced over the details today for this post. It
shouldn't matter too much tho for the purpose of this post since we're mostly
focusing on the end-user experience.</p>
</div>}func </span>makeDinner() async throws -> Meal {
</span>  async </span>let</span> veggies = chopVegetables()
</span>  async </span>let</span> meat = marinateMeat()
</span>  async </span>let</span> oven = preheatOven(temperature: </span>350</span>)
</span>
</span>  </span>let</span> dish = Dish(ingredients: await [</span>try</span> veggies, meat]) </span>// notice the concurrent await here
</span>  </span>return try</span> await oven.cook(dish, duration: .hours(</span>3</span>))
</span>}
</span></code></pre>
This feels very elegant in comparison, and definitely feels like a step up for
Swift. I think the flow of this is so good in fact, that we may want to adopt
something similar in Rust.</p>
Join in Rust</h2>
In Rust we already have a notion of a "concurrent await" through the join</code>
operation. If you do async Rust you've likely seen it before in macro-form as
join!</code></a> or the join_all</code></a> free-function:</p>
// example of the `join!` macro:
</span>let</span> a = future::ready(</span>1</span>u8</span>);
</span>let</span> b = future::ready("</span>hello</span>");
</span>let</span> c = future::ready(</span>3</span>u16</span>);
</span>assert_eq!(join!(a, b, c).await, (</span>1</span>, "</span>hello</span>", </span>3</span>));
</span>
</span>// example of the `join_all` free-function:
</span>let</span> futs = vec![</span>ready</span>(</span>1</span>u8</span>), </span>foo</span>(</span>2</span>u8</span>), </span>foo</span>(</span>3</span>u8</span>)];
</span>assert_eq!(</span>join_all</span>(futs).await, [</span>1</span>, </span>2</span>, </span>3</span>]);
</span></code></pre>
In my Futures Concurrency II</a> 3</a></sup> post and library</a> I show that instead of using a
combination of macros and free functions we can instead use traits to extend
container types with the necessary concurrency operations. This results in a
smoother experience overall, since arrays, tuples, and vectors all operate in
the same way:</p>
^3</sup>Y. Wuyts, “Futures Concurrency II,” Sep. 02, 2021</a>. (accessed Apr. 11, 2022).</p>
</div>
// alternative to the `join!` macro:
</span>let</span> a = future::ready(</span>1</span>u8</span>);
</span>let</span> b = future::ready("</span>hello</span>");
</span>let</span> c = future::ready(</span>3</span>u16</span>);
</span>assert_eq!((a, b, c).</span>join</span>().await, (</span>1</span>, "</span>hello</span>", </span>3</span>));
</span>
</span>// alternative to the `join_all` free-function:
</span>let</span> a = future::ready(</span>1</span>);
</span>let</span> b = future::ready(</span>2</span>);
</span>let</span> c = future::ready(</span>3</span>);
</span>assert_eq!(vec![a, b, c].</span>join</span>().await, vec![</span>1</span>, </span>2</span>, </span>3</span>]);
</span></code></pre>
This works fine, but it's not quite as good as what swift has using async let</code>.
Let's take a look at how we can improve that.</p>
Join Ergonomics</h2>
Awaiting a single future is done by calling .await</code>, but awaiting multiple
futures requires calling an intermediate method. To get join</code> semantics you have
to call the join</code> method, rather than just being able to await a tuple of futures.</p>
This is where IntoFuture</code> can help: we can use it to implement a conversion
to a future directly on tuples, arrays, and vectors - and make it so if they
contain futures you can just call .await</code> on them directly to await all futures
concurrently. With that in place the above example could be rewritten like this:</p>
let</span> a = future::ready(</span>1</span>u8</span>);
</span>let</span> b = future::ready("</span>hello</span>");
</span>let</span> c = future::ready(</span>3</span>u16</span>);
</span>assert_eq!((a, b, c).await, (</span>1</span>, "</span>hello</span>", </span>3</span>));    </span>// no more `.join()` needed
</span>
</span>let</span> a = future::ready(</span>1</span>);
</span>let</span> b = future::ready(</span>2</span>);
</span>let</span> c = future::ready(</span>3</span>);
</span>assert_eq!(vec![a, b, c].await, vec![</span>1</span>, </span>2</span>, </span>3</span>]);  </span>// no more `.join()` needed
</span></code></pre>
While being substantially different under the hood, the surface-level experience of
this is actually quite similar to Swift's async let</code>. We just don't operate by
default on tasks owned by a runtime, but futures which are lazy and compile down
to in-line state machines. But this could be made to trivially work with
multi-threaded Rust tasks too, if we follow the spawn approach laid out by
tasky</a> (blog post</a> 4</a></sup>) 5</a></sup>:</p>
^4</sup>Y. Wuyts, “Postfix Spawn,” Mar. 04, 2022</a>. (accessed Sep. 18, 2022).</p>
</div>
^{5</sup>
I should explain this model in more detail at some point. The
core of it is that we treat parallelism as a resource, and concurrency as a way
of scheduling work. We mark futures which are "parallelizable" as such, and then
pass them to the existing concurrency operators just like any other future. That
creates a unified approach to concurrency for both parallel and non-parallel
workloads.</p>
</div>}let</span> a = future::ready(</span>1</span>u8</span>).</span>spawn</span>();
</span>let</span> b = future::ready("</span>hello</span>").</span>spawn</span>();
</span>let</span> c = future::ready(</span>3</span>u16</span>).</span>spawn</span>();
</span>assert_eq!((a, b, c).await, (</span>1</span>, "</span>hello</span>", </span>3</span>));    </span>// parallelized `await`
</span>
</span>let</span> a = future::ready(</span>1</span>).</span>spawn</span>();
</span>let</span> b = future::ready(</span>2</span>).</span>spawn</span>();
</span>let</span> c = future::ready(</span>3</span>).</span>spawn</span>();
</span>assert_eq!(vec![a, b, c].await, vec![</span>1</span>, </span>2</span>, </span>3</span>]);  </span>// parallelized `await`
</span></code></pre>
This, to me, feels like a pretty good outcome for concurrent and parallel
awaiting of fixed-sized sets of futures. Awaiting sets of futures which change
over time is a different problem, and will likely require something akin to
Swift's TaskSet</code> to function. But that's not most concurrent workloads, and I
think we can take a page out of Swift's book on this.</p>
Other Operations</h2>
In a past post</a> 6</a></sup> I've shown the following table of
concurrency operations for futures:</p>
^6</sup>Y. Wuyts, “Futures Concurrency III: select,” Feb. 09, 2022</a>. (accessed Apr. 06, 2022).</p>
</div>
</th> Wait for all outputs</strong></th> Wait for first output</strong></th></tr></thead>

Continue on error</strong></td> Future::join</code></td> Future::try_race</code></td></tr>
Return early on error</strong></td> Future::try_join</code></td> Future::race</code></td></tr>
</tbody></table>
join</code> is only one of 4 different concurrency operations. When you have
different futures you may in fact want to do different things with it. But not
all operations are equal: when we have two or more futures, it's usually because
we're interested in observing all of their output. join</code> gives us exactly that.
We'll get to try_</code> variants in the next section, but for now take for the
following example:</p>
let</span> a = future::ready(</span>1</span>);
</span>let</span> b = future::ready(</span>2</span>);
</span>let</span> a = a.await;
</span>let</span> b = b.await;
</span></code></pre>
This first awaits a</code>, and then it awaits b</code>. Because the two are unrelated,
it's often more efficient to await them concurrently</em> rather than
sequentially</em>. With (async) IntoIterator</code> we're handed tools required to operate
over values sequentially. But with IntoFuture</code> we're handed tools to operate
over values concurrently</em> ^{7</a></sup>:</p>}
^{7</sup>
IntoIterator</code> is not implemented for tuples, only for arrays and
vectors. To make this example work we'll just use arrays. Perhaps we can one day
use tuples too like this, but that may require having variadic tuples and maybe
even structurally-typed enums for it to work.</p>
</div>}// `IntoIterator` enables sequential operation
</span>let</span> a = </span>1</span>;
</span>let</span> b = </span>2</span>;
</span>for</span> num in [a, b] { .. }   </span>// sequential iteration
</span>
</span>// `IntoFuture` enables concurrent operation
</span>let</span> a = future::ready(</span>1</span>);
</span>let</span> b = future::ready(</span>2</span>);
</span>let </span>[a, b] = [a, b].await; </span>// concurrent await
</span></code></pre>
The design of IntoFuture</code> intentionally mirrors the design of IntoIterator</code>.
For any type T</code> we can only have a single IntoIterator</code> implementation. This
leaves us with a few options:</p>

don't implement IntoFuture</code> for containers</li>
implement join</code> semantics for containers</li>
implement race</code> semantics for containers</li>
</ol>
This post is argues that 2.</code> is a chance to provide better ergonomics than
1.</code>. But what about 3.</code>? Could we meaningfully have race</code> semantics as the
default? race</code> takes N futures and yields one result. This would look like
this:</p>
let</span> a = future::ready(</span>1</span>u8</span>);
</span>let</span> b = future::ready(</span>2</span>u8</span>);
</span>let</span> c: </span>u8 </span>= [a, b].await; </span>// `race`-semantics
</span></code></pre>
I've said it before, but this seems like a minority use case. We'll almost always
want a AND b</code>. Not a XOR b</code>. At least: we may still want to drop the values
later on, but it's usually not decided based on which completed first. join</code>
also feels like it extends the existing behavior of await</code> for individual
futures to sets of futures. race</code> exposes different semantics than a plain
.await</code> does, and so if awaiting tuples of futures was different, the
resulting experience would feel inconsistent.</p>
It seems good to have join</code> be the default semantics exposed through
IntoFuture</code>, but then have operations such as race</code> and merge</code> be explicit
forms of concurrency you can instead opt into by calling the right method.</p>
Fallibility</h2>
There is one other issue with the proposal: we have no way to expose fallbility
of try_join</code> semantics.  try_join</code> allows short-circuiting a join</code> operation
if any of the fallible futures returns a Result::Error</code>. Right now we don't have a way
to paramaterize the await</code> over fallibility, and since we only have a single
IntoFuture</code> implementation we're stuck with just join</code>, which is not ideal.
Perhaps using keyword generics</a> 8</a></sup> we can find a way out of this.
After all: iterators face many of the same challenges today, since we can't
just call for ?..in</code> in loops. With keyword generics we might be able to
choose try_join</code> semantics for constructs such as:</p>
^8</sup>Y. Wuyts, “Announcing the Keyword Generics Initiative”</a>. (accessed Sep. 18, 2022).</p>
</div>
let</span> a = future::ready(Ok(</span>1</span>));
</span>let</span> b = future::ready(Ok(</span>2</span>));
</span>let </span>(a, b) = (a, b).await?; </span>// try_join await semantics
</span></code></pre>
This would infer that the want the fallible veriant of IntoFuture</code> since we
call ?</code> on it after, and we're in a fallible context. But it's early on and
hard to say precisely how this would work. It seems encouraging though that
wanting fallible semantics for a construct is a shared problem across most of
the language, so there is a desire to solve it in a unified way.</p>
Conclusion</h2>
In this post I've shown how Swift's async let</code> allows concurrent awaiting, how
we can implement it in Rust through IntoFuture</code>, and what some of the
challenges with it might be. I think Swift has done a great job at showing how
convenient it is to be able to perform lightweight concurrent awaits, and I
think we can learn from their approach and make it our own.</p>
Before starting this post I didn't think too deeply about the similarities
between IntoIterator</code> and IntoFuture</code>. But the more I look at them, the closer
they seem. The fact that we have IntoIterator</code> for most every container might give
us a hint for how we may want to think about container types. I'm not saying we
should go out and implement IntoFuture</code> for HashMap</code> - but for vec, array, and
tuples it definitely seems to make sense.</p>
It seems that in terms of ordering we aren't too far out from being able to
write an RFC for all of this either. I'm feeling pretty confident we've got the
basic subset done for fixed-length async concurrency operators. And with
IntoFuture</code> exposing join</code> semantics for arrays, tuples, and vectors, we can
make the main method of concurrency easy to use as well. We'll have to see
exactly what we want to do about tuples wrt the design - since they can't yet be
variadic. And we'll want to wait on async traits to land first. But after 3
years of writing and experimentation on this topic, we don't seem far out
9</a></sup> from finally being able to propose something concrete for
inclusion in the stdlib.  And I think that's pretty exciting!</p>
^9</sup>The join!</code>
macro</a> was added to the
stdlib a while back through a regular libs PR. But any stabilization of it
should be blocked on fleshing out a complete concurrency model for async
Rust</a>.</p>
</div>

State Machines II: an enum-based design
Tue, 23 Aug 2022 00:00:00 +0000
In the past I've written about state machines</a> in Rust. And more recently I've
also written about anonymous enums</a> 1</a></sup> in Rust. I've been wondering what would
happen if the two interacted? I think the result could actually be quite nice,
and worth daydreaming about. In this post we'll be discussing state machines,
the language features which could make them easier to use, and ways in which we
could push the ergonomics further.</p>
^{1</sup>
"anonymous enums" are also known as "sum types". There might be
subtle differences, but I'll be using them interchangeably. For
the context of this post though, we assume that taking two members
of the same enum and putting them in a sum type results in a
sub-set of the original enum rather than creating a new enum with
two members. For the sake of brevity in this post, just assume
that works as intended.</p>
</div>}The case for state machines</h2>
A lot of programming is about going from one state to another. We can do that
using conditionals statements, but as the state changes those conditions can be
tricky to reason about and in turn refactor. Instead it's often more convenient
to use state machines</em>: singular objects which represent all possible states
and their transitions. And in compiled languages specifically to use
compiler-checked state machines</em> 2</a></sup>.</p>
^2</sup>See my JavaScript
nanostate</a> library for an example of a
state machine library which is not checked by a compiler - instead performing
all checks at runtime.</p>
</div>
Say we wanted to implement a basic green-orange-red traffic light that can also
flash. We can imagine the rules for transitions between states could be the
following:</p>

Green can turn to orange</li>
Orange can turn to red</li>
Red can turn to green</li>
Red can turn to flashing red 3</a></sup></li>
Flashing red can turn to red</li>
</ol>
^{3</sup>
If you've never seen a traffic light blink red before:
this is common for traffic lights in the Netherlands. It can indicate that the
traffic control systems are malfunctioning. Or it can be used to indicate the
traffic lights have been disabled, which can be done late at night when there's
hardly any traffic to be managed anyway.</p>
</div>
All states and their transitions can be represented by the following graph:</p>
</p>
For the remainder of this post we'll be implementing the rules of the graph
above. Our goal is to implement a compiler-checked state machine which can
inform us during compilation whether all our state transitions are valid, or if
we've accidentally written an invalid transition which we need to fix.</p>}Representing State Machines in Rust In The Future</h2>
To read about how we can write compiler-checked state machines in Rust today</em> you
can read my previous post on state machines</a>. In this post I instead want to focus on
✨ the future ✨. How could we plausibly represent this in Rust by extending the
language. In the "future directions" section of the last post we wrote a state
machine without the blinky lights. Using enum variants types</a> (not yet a
thing) combined with arbitrary self-types</a> (also not
yet a thing), we could write the following:</p>
//! A traffic light without a flashing state.
</span>//! Novel language features: enum-variant-types, arbitrary-self-types
</span>
</span>enum </span>State {
</span>    Green,
</span>    Orange,
</span>    Red,
</span>}
</span>
</span>impl </span>State {
</span>    </span>pub fn </span>new</span>() -> </span>Self::</span>Green {
</span>        </span>Self</span>::Green
</span>    }
</span>
</span>    </span>pub fn </span>next</span>(</span>self</span>: </span>Self::</span>Green) -> </span>Self::</span>Orange {
</span>        </span>Self</span>::Orange
</span>    }
</span>
</span>    </span>pub fn </span>next</span>(</span>self</span>: </span>Self::</span>Orange) -> </span>Self::</span>Red {
</span>        </span>Self</span>::Red
</span>    }
</span>
</span>    </span>pub fn </span>next</span>(</span>self</span>: </span>Self::</span>Red) -> </span>Self::</span>Green {
</span>        </span>Self</span>::Green
</span>    }
</span>}
</span>
</span>fn </span>main</span>() {
</span>    </span>let mut</span> state = State::new(); </span>// green
</span>    state = state.</span>next</span>();         </span>// orange
</span>    state = state.</span>next</span>();         </span>// red
</span>    state = state.</span>next</span>();         </span>// green
</span>}
</span></code></pre>
What I didn't realize at the time I wrote my last post is that
this in effect acts like a form of overloading</em>. Functionally the code we've
written isn't different than writing a single next</code> function which internally
updates the state. But instead we expose the state externally</em>. The following
example will be written in valid Rust today</em>, exposing the exact same
functionality as our last example — but with the state only observable
internally</em>:</p>
//! A traffic light without a flashing state.
</span>//! Novel language features: none
</span>
</span>enum </span>State {
</span>    Green,
</span>    Orange,
</span>    Red,
</span>}
</span>
</span>impl </span>State {
</span>    </span>pub fn </span>new</span>() -> </span>Self::</span>Green {
</span>        </span>Self</span>::Green
</span>    }
</span>
</span>    </span>pub fn </span>next</span>(</span>self</span>) -> </span>Self </span>{
</span>        </span>match </span>self </span>{
</span>            </span>Self</span>::Green => </span>Self</span>::Orange,
</span>            </span>Self</span>::Orange => </span>Self</span>::Red,
</span>            </span>Self</span>::Red => </span>Self</span>::Green,
</span>        }
</span>    }
</span>}
</span>
</span>fn </span>main</span>() {
</span>    </span>let mut</span> state = State::new(); </span>// green
</span>    state = state.</span>next</span>();         </span>// orange
</span>    state = state.</span>next</span>();         </span>// red
</span>    state = state.</span>next</span>();         </span>// green
</span>}
</span></code></pre>
Let's continue with the internally observable state examples for a bit. Our
graph shows we need to support a flashing</code> state, so we need to add that.
We need some way to trigger going into a flashing state, so we'll add a new
method: flash</code>. Not all states can transition into the flashing state, so we
need to make sure we can only go into it from "red" - which means we need to
panic if we attempt to transition into it from any other state:</p>
//! A traffic light with a flashing state.
</span>//! Novel language features: none
</span>
</span>enum </span>State {
</span>    Green,
</span>    Orange,
</span>    Red,
</span>    Flashing,
</span>}
</span>
</span>impl </span>State {
</span>    </span>pub fn </span>new</span>() -> </span>Self::</span>Green {
</span>        </span>Self</span>::Green
</span>    }
</span>
</span>    </span>pub fn </span>next</span>(</span>self</span>) -> </span>Self </span>{
</span>        </span>match </span>self </span>{
</span>            </span>Self</span>::Green => </span>Self</span>::Orange,
</span>            </span>Self</span>::Orange => </span>Self</span>::Red,
</span>            </span>Self</span>::Red => </span>Self</span>::Green,
</span>            </span>Self</span>::Flashing => </span>Self</span>::Red,
</span>        }
</span>    }
</span>
</span>    </span>pub fn </span>flash</span>(</span>self</span>) -> </span>Self </span>{
</span>        </span>match </span>self </span>{
</span>            State::Red => </span>Self</span>::Flashing
</span>            _ => panic!("</span>only red can turn into flashing red</span>"),
</span>        }
</span>    }
</span>}
</span>
</span>fn </span>main</span>() {
</span>    </span>let mut</span> state = State::new(); </span>// green
</span>    state = state.</span>next</span>();         </span>// orange
</span>    state = state.</span>next</span>();         </span>// red
</span>    state = state.</span>flash</span>();        </span>// flashing
</span>    state = state.</span>next</span>();         </span>// red
</span>}
</span></code></pre>
This is not ideal: we now have added a runtime check to a state machine which
was checked entirely during compilation before. The reason a runtime check
is needed is because by adding a method which isn't available on all types,
the internal</em> state becomes observable externally</em>. And the Rust language
lacks the tools to conveniently represent this 4</a></sup>.</p>
^{4</sup>
Yes, I know we can refactor the whole thing and use traits and
generics to hack together something which gives us similar semantics. I've shown
how to do that in previous posts. The point of this post is not "can we
represent state machines using Rust's type system"</em> (yes we can), but "which
language features are we lacking to make it easy to represent state machines in
Rust's type system"</em>.</p>
</div>
So let's try and rewrite this again using enum variant types and arbitrary
self-types. Here the caller knows exactly what state the enum is in, and we can
more precisely represent the states and state transitions:</p>}//! A traffic light with a flashing state.
</span>//! Novel language features: enum-variant-types, arbitrary-self-types
</span>
</span>enum </span>State {
</span>    Green,
</span>    Orange,
</span>    Red,
</span>    Flashing,
</span>}
</span>
</span>impl </span>State {
</span>    </span>pub fn </span>new</span>() -> </span>Self::</span>Green {
</span>        </span>Self</span>::Green
</span>    }
</span>
</span>    </span>pub fn </span>next</span>(</span>self</span>: </span>Self::</span>Green) -> </span>Self::</span>Orange {
</span>        </span>Self</span>::Orange
</span>    }
</span>
</span>    </span>pub fn </span>next</span>(</span>self</span>: </span>Self::</span>Orange) -> </span>Self::</span>Red {
</span>        </span>Self</span>::Red
</span>    }
</span>
</span>    </span>pub fn </span>next</span>(</span>self</span>: </span>Self::</span>Red) -> </span>Self::</span>Green {
</span>        </span>Self</span>::Green
</span>    }
</span>
</span>    </span>// new
</span>    </span>pub fn </span>next</span>(</span>self</span>: </span>Self::</span>Flashing) -> </span>Self::</span>Red {
</span>        </span>Self</span>::Red
</span>    }
</span>
</span>    </span>// new
</span>    </span>pub fn </span>flash</span>(</span>self</span>: </span>Self::</span>Red) -> </span>Self::</span>Flashing {
</span>        </span>Self</span>::Flashing
</span>    }
</span>}
</span>
</span>fn </span>main</span>() {
</span>    </span>let mut</span> state = State::new(); </span>// green
</span>    state = state.</span>next</span>();         </span>// orange
</span>    state = state.</span>next</span>();         </span>// red
</span>    state = state.</span>flash</span>();        </span>// flashing
</span>    state = state.</span>next</span>();         </span>// red
</span>}
</span></code></pre>
This would allow us to correctly model and validate the entire state graph at
compile-time, without needing to resort to any runtime checks. Yay us!</p>
Ergonomics: Overview</h2>
Hopefully folks can see the benefit of being able to model state transitions
solely through the type system. Obviously there are other ways of authoring
compiler-checked state machines today - but that's not the point of this post.
What we're trying to think about is how we can do so in a way which feels
simple</em> and dare I even say, intuitive</em> for people to use.</p>
I think what we've shown so far is pretty good. But it has a few shortcomings,
namely:</p>

It's pretty verbose to individually write each transition</li>
What happens when we have more than one valid return state from a function?</li>
</ol>
Okay, so bear with me for a sec here. In the graph each line goes from a source</em>
to a destination</em>. In the code the sources are represented by the self-types,
and the destinations are represented by the return types. This means that any
node in the graph with more than arrow pointing from or to it could potentially
benefit from refactoring. In the graph above there are instances of this:</p>

orange</code> and flashing</code> point to red</code></li>
red</code> points to both green</code> and flashing</code></li>
</ol>
For any type system enthusiasts reading this post: yes we're going to be
talking about representing powerset enums natively using Rust's typesystem
here 5</a></sup>.</p>
^5</sup>For everyone who isn't a type system enthusiast and doesn't know
about "powerset enums", it's basically just "subsets of enums". Instead of
declaring that a method can return all</em> variants in an enum, using powersets we
can declare that a method can only return a specific subset. Check out the
powerset_enum crate</a> to read more about it. Or alternatively you can keep
reading, because that's basically what we'll be implementing.</p>
</div>
Ergonomics: Merging Self-Types</h2>
Now let's start with the first one: "orange" and "flashing" both point to "red". So
far we've been talking about arbitrary self-types as a feature. But there are
ideas floating around about expanding this for structs, called: view types</a>.
What this allows you to do is inform the borrow checker we don't actually need
access to the entire struct - we only need access to parts of it. So we can hold
multiple mutable borrows on the same struct as long as the fields don't overlap.</p>
Now what if we had the equivalent of these struct "views" but for enums. Instead
of "viewing" subsets of fields, we'd be able to "view" subsets of enums. Now
let's make up a syntax to show what we mean. In patterns we already have Foo | Bar</code> to represent multiple patterns, so let's reuse that here:</p>
// before
</span>impl </span>State {
</span>    </span>pub fn </span>next</span>(</span>self</span>: </span>Self::</span>Orange) -> </span>Self::</span>Red {
</span>        </span>Self</span>::Red
</span>    }
</span>
</span>    </span>pub fn </span>next</span>(</span>self</span>: </span>Self::</span>Flashing) -> </span>Self::</span>Red {
</span>        </span>Self</span>::Red
</span>    }
</span>}
</span></code></pre>
// after
</span>impl </span>State {
</span>    </span>pub fn </span>next</span>(</span>self</span>: </span>Self::</span>Orange | </span>Self::</span>Flashing) -> </span>Self::</span>Red {
</span>        </span>Self</span>::Red
</span>    }
</span>}
</span></code></pre>
I know many of you will likely have opinions about the syntax, but let's not
focus on that for a minute. Instead I want to focus your attention attention on
the semantics at play here: it would bring the same pattern semantics as used in
match</code> statements to methods, which I believe should feel a lot more natural?</p>
Riffing time. There are some real questions about what syntax view types should use</a>.
Niko's post suggests something like &{golden_tickets} self</code>. But what if
instead we chose to reuse the pattern syntax for the self</code> types 6</a></sup> -
which could yield us something like &Self { golden_tickets, .. }</code>. This would
reuse the pattern-destructuring syntax. Which is similar to what we're doing
here: reusing the pattern enum matching syntax. It feels like there might be
something to it?</p>
^6</sup>My colleague eric</a> suggested
exploring this direction a while back. And I actually quite like it! (Direct any
credit to Eric, but blame to me please. I've definitely run with this way past
what he likely intended.)</p>
</div>
The compiler knows that the possible members of self</code> in the example ("orange",
"flashing") are only a subset of all members in State</code>. What if the compiler
actually used this information in the function body too? That would mean matching
on self</code> would only need to account for the possible</em> cases rather than all</em>
cases:</p>
match </span>self </span>{
</span>    </span>Self</span>::Orange => {}
</span>    </span>Self</span>::Flashing => {}
</span>    </span>// no other cases need matching!
</span>}
</span></code></pre>
This isn't particularly relevant yet for the simple traffic light example we're
dealing with here, but definitely becomes a lot more relevant when working with
real-world code. No longer needing to panic!</code> for branches you know can't be
reached anyway would make code both simpler and faster, which is a pretty big deal!</p>
Ergonomics: Merging Return Types</h2>
The second case is: "red can point to both green and flashing"</em>. This one will be a bit
harder to reason about because instead of taking multiple different parameters
we're trying to return</em> multiple parameters. This means the compiler can
no longer statically</em> know which variant we have, and instead we have to
perform runtime matching to figure that out. Right now the code we have looks
like this:</p>
// before
</span>impl </span>State {
</span>    </span>pub fn </span>next</span>(</span>self</span>: </span>Self::</span>Red) -> </span>Self::</span>Green {
</span>        </span>Self</span>::Green
</span>    }
</span>
</span>    </span>pub fn </span>flash</span>(</span>self</span>: </span>Self::</span>Red) -> </span>Self::</span>Flashing {
</span>        </span>Self</span>::Flashing
</span>    }
</span>}
</span></code></pre>
We have two different methods which take Self::Red</code> and return a type. For
argument's sake say we wanted a single method with took some data to determine
whether we should go to green</code> or flashing</code> 7</a></sup>. This could then look
something like this:</p>
^{7</sup>
"For argument's sake" means I couldn't come up with a good example. So
this is def a cop-out.</p>
</div>}// after
</span>impl </span>State {
</span>    </span>pub fn </span>next</span>(</span>self</span>: </span>Self::</span>Red, </span>data</span>: Data) -> </span>Self::</span>Green | </span>Self</span>::Flashing {
</span>        </span>if </span>validate_data</span>(data) {
</span>            </span>Self</span>::Green
</span>        } </span>else </span>{
</span>            </span>Self</span>::Flashing
</span>        }
</span>    }
</span>}
</span></code></pre>
Again, the syntax is not really the point. The idea here is though that just
like in the earlier example the returned value would be known to be a subset of
Self</code>: it would still register as the State</code> enum, but the only two accessible
members are Green</code> and Flashing</code> 8</a></sup>.</p>
^{8</sup>
Maybe a better way to signal this would be to use something like
-> Self { Green | Flashing }</code> as syntax? Idk,, I'm focusing more on semantics than
syntax right now.</p>
</div>
On the usage side things would change too: we could no longer statically know
which state we're in. All we know is that the state is either green or flashing.
This leads to some very real questions: should we be able to then call next</code>
again to get to orange | red</code>?</p>}fn </span>main</span>() {
</span>    </span>let mut</span> state = State::new(); </span>// green
</span>    state = state.</span>next</span>();         </span>// orange
</span>    state = state.</span>next</span>();         </span>// red
</span>    state = state.</span>next</span>(data);     </span>// green | flashing       - ok I guess?
</span>    state = state.</span>next</span>();         </span>// orange | red           - uhhh,,
</span>    state = state.</span>next</span>(?data);    </span>// green | red | flashing - x_x
</span>}
</span></code></pre>
Once we're in this state, we have different next</code> methods. How do we decide how
we should call them? Even if we can answer this question, the semantics of this
seem pretty complex and hard to teach. So instead I propose we apply some
basic rules here to simplify usage:</p>

Once a variable is assigned to a single enum-member-as-value, it can only be
re-assigned to other single values and vice-versa.9</a></sup></li>
Methods with the same name on the same enum need to have the same "base"
(type from which a view is derived) signature.</li>
</ol>
^{9</sup>
Maybe this rule could be relaxed over time? I'm not sure; it seems very
graph-problem-y, and I don't know if there are any great solutions to this.</p>
</div>
The first rule would disallow assigning green | flashing</code> to state</code>. Instead
we'd need to match</code> to know which variant we have, and go from there:</p>}//! Never assign an enum view-type to `state`. This requires matching on the data
</span>//! until we've extraced a single member. Assume we wait for conditions to change
</span>//! between calls to `state.next`.
</span>
</span>fn </span>main</span>() {
</span>    </span>let mut</span> state = State::new();                    </span>// green
</span>    state = state.</span>next</span>();                            </span>// orange
</span>    state = state.</span>next</span>();                            </span>// red
</span>    state = </span>handle_flash</span>(state.</span>next</span>(&data), &data);  </span>// green
</span>}
</span>
</span>/// Keep looping until we hit a green state.
</span>fn </span>handle_flash</span>(</span>mut </span>state</span>: </span>State::</span>Green | </span>State::</span>Flashing, </span>data</span>: &Data) -> </span>State::</span>Green {
</span>    </span>match</span> state {
</span>        State::Green => State::Green,
</span>        State::Flashing => {
</span>            state = state.</span>next</span>();              </span>// red
</span>            </span>to_green</span>(state.</span>next</span>(&data), &data) </span>// green
</span>        }
</span>    }
</span>
</span>}
</span></code></pre>
There are a bunch of assumptions baked in here - not in the least that an anonymous enum of
State::Green | State::Flashing</code> can be passed to an enum view-type matching the
same members. But we're not doing details in this post. We're doing vibes. Yeehaw.</p>
Okay, so for the second rule we're proposing: methods with the same name on the same enum
should share the same base signature</em>. This means that methods which return a
view</em> of some enum or struct are fine - because they're the same "base" type.
But having different methods with different signatures like we've done here with
next(data)</code> would be a no-no. This would allow us to keep thinking about
methods which take or return enum views as a form of specialization.</p>
That last point is perhaps not strictly required, but it has the benefit of
making the enum feel more consistent</em>. And it could lead allow for
specialization-like behavior where if you implement the same method for all enum
members, it then becomes unconditionally available on the entire enum - since it
can be called regardless of which state you're in, even if it yields different
behavior. But on a more intuitive level: it feels pretty weird that
self.next()</code> sometimes takes an argument and sometimes doesn't. It kind of
feels like parameter overloading like it's done in other languages, and Rust
kind of steers clear of that. So it feels better to just enforce that methods
with the same name on the same enum need to take the same arguments - even if we
can wiggle around with view-types and sub-types a little.</p>
Tying It Together</h2>
Phew, that's been a ride. The last section has been a lot about what not</em> to
do. So let's show an example of how things might look when we put them all
together. Let's start with what we had before, but slightly adapted to make the
flash</code> case conditional:</p>
//! A traffic light with a flashing state.
</span>//! Novel language features: enum-variant-types, arbitrary-self-types
</span>
</span>enum </span>State {
</span>    Green,
</span>    Orange,
</span>    Red,
</span>    Flashing,
</span>}
</span>
</span>impl </span>State {
</span>    </span>pub fn </span>new</span>() -> </span>Self::</span>Green {
</span>        </span>Self</span>::Green
</span>    }
</span>
</span>    </span>pub fn </span>next</span>(</span>self</span>: </span>Self::</span>Green) -> </span>Self::</span>Orange {
</span>        </span>Self</span>::Orange
</span>    }
</span>
</span>    </span>pub fn </span>next</span>(</span>self</span>: </span>Self::</span>Orange) -> </span>Self::</span>Red {
</span>        </span>Self</span>::Red
</span>    }
</span>
</span>    </span>pub fn </span>next</span>(</span>self</span>: </span>Self::</span>Red) -> </span>Self::</span>Green {
</span>        </span>Self</span>::Green
</span>    }
</span>
</span>    </span>// new
</span>    </span>pub fn </span>next</span>(</span>self</span>: </span>Self::</span>Flashing) -> </span>Self::</span>Red {
</span>        </span>Self</span>::Red
</span>    }
</span>
</span>    </span>// new
</span>    </span>pub fn </span>flash</span>(</span>self</span>: </span>Self::</span>Red) -> </span>Self::</span>Flashing {
</span>        </span>Self</span>::Flashing
</span>    }
</span>}
</span>
</span>fn </span>main</span>() {
</span>    </span>let mut</span> state = State::new(); </span>// green
</span>    state = state.</span>next</span>();         </span>// orange
</span>    state = state.</span>next</span>();         </span>// red
</span>    </span>if </span>validate_data</span>(data) {
</span>            state = </span>self</span>.</span>next</span>();  </span>// green
</span>            </span>// handle green
</span>        } </span>else </span>{
</span>            state = </span>self</span>.</span>flash</span>(); </span>// flashing
</span>            </span>// handle flashing
</span>        }
</span>    }
</span>}
</span></code></pre>
Now if we translate that to our enum views variant, we're able to inline the
conditional statement into the enum. And together with the arbitrary self enum
pattern syntax, the overall code is a lot more trimmed. And especially the usage
should feel a bit more natural:</p>
//! A traffic light with a flashing state.
</span>//! Novel language features: enum-variant-types, arbitrary-self-types,
</span>//!                          enum-view-types, enum-sum-types
</span>
</span>enum </span>State {
</span>    Green,
</span>    Orange,
</span>    Red,
</span>    Flashing,
</span>}
</span>
</span>impl </span>State {
</span>    </span>pub fn </span>new</span>() -> </span>Self::</span>Green {
</span>        </span>Self</span>::Green
</span>    }
</span>
</span>    </span>pub fn </span>next</span>(</span>self</span>: </span>Self::</span>Green) -> </span>Self::</span>Orange {
</span>        </span>Self</span>::Orange
</span>    }
</span>
</span>    </span>pub fn </span>next</span>(</span>self</span>: </span>Self::</span>Orange | </span>Self::</span>Flashing) -> </span>Self::</span>Red {
</span>        </span>Self</span>::Red
</span>    }
</span>
</span>    </span>// New: Inlines the logic of the previous example to decide whether to go
</span>    </span>//      from "red" to "green" or "flashing".
</span>    </span>pub fn </span>try_next</span>(</span>self</span>: </span>Self::</span>Red, </span>data</span>: Data) -> </span>Self::</span>Green | </span>Self</span>::Flashing {
</span>        </span>if </span>validate_data</span>(data) {
</span>            </span>Self</span>::Green
</span>        } </span>else </span>{
</span>            </span>Self</span>::Flashing
</span>        }
</span>    }
</span>}
</span>
</span>fn </span>main</span>() {
</span>    </span>let mut</span> state = State::new();   </span>// green
</span>    state = state.</span>next</span>();           </span>// orange
</span>    state = state.</span>next</span>();           </span>// red
</span>    state.</span>try_next</span>(data) {
</span>        State::Green => {},         </span>// handle green
</span>        State::Flashing => {},      </span>// handle flashing
</span>    };
</span>}
</span></code></pre>
Conclusion</h2>
In this post we've looked at how using "enum variant types", "enum sum
types", and "enum view types" could allow us to make state machines at the type
level in Rust easier to author and more ergonomic. The example we've used here
has been a variation of a basic traffic light. But despite that we expect that
it should be possible to adopt the patterns and features shown here to state
machines in general.</p>
Challenges</strong></p>
It seems that taking enums as self-types should be relatively straight-forward.
But returning anonymous enums is more complicated. I came up with some rules to
help circumvent some of the sharp edges, but I feel those probably could use
some more thought. Maybe folks reading this have some insights on it?</p>
Making Enums More Useful</strong></p>
Also as my colleague Arlie</a> pointed out during a recent chat: having
enums-as-types would be really beneficial for other reasons too. Take for
example the syn</code> crate 10</a></sup>: the WherePredicate</code></a> has a member Type</code> which
contains a struct called PredicateType</code></a>. The reason why we have a separate
struct instead of just inlining the data in the enum member, is because members
can't be used as types. So if we want to author functions which operate on the
data in WherePredicate::Type</code>, we need</em> to create a separate struct 11</a></sup>.
This makes record-enums less useful than their struct counterparts, and
complicates the API surface area of crates.</p>
^{10</sup>
I picked this as an example, so if there are actually different
instances where PredicateType</code> is used - that's on me. I'm not an expert on
syn</code> so details may be wrong. But I hope the example is clear enough that at
least the point comes across: because this does happen in a lot of places.</p>
</div>}^{11</sup>
Ideally we should only create need to create separate structs if we
want to reuse types between enums. That's not the case today.</p>
</div>
Structural Typing and Errors</strong></p>}While we're on the topic of anonymous enums in general: another important use
case for it would be for error handling. Anonymous enums provide us with
structural typing for enums 12</a></sup>, which allows for convenient
composition of error types. Say we have a ParseError</code></a> and io::Error</code></a>, a
function should just be able to specify -> Result<T, ParseError | io::Error></code>.
But more importantly: we should be able to specify which error variants can be
returned. This is a bit harder with io::Error</code> specifically because of how it's
constructed. But would work great when using errors returned by thiserror</code></a>.
I've been meaning to write about error handling ever since I posted my error
handling survey</a> three years ago. Maybe I'll do that later this year.</p>
^{12</sup>
You can think of "structural typing" as "matching on the shape"
instead of "matching on the type". Matching on a tuple with (first, second)</code> is
structural: we don't care what types are in the tuple, we mostly care about the
shape. It still needs to type-check of course, but combined with type inference
it can give us a lot of flexibility. For example TypeScript's type system is
structural, and in turn provides a lot of flexibility.</p>
</div>
View Types</strong></p>}Also, while writing this post I realized that being able to use enum variants
as self-types is kind of analogous to view types for structs. Both carve out
subsets of their base type, but for different purposes. Which makes sense
because they're actually different. But as a general trend the implications seem
fun: by more precisely articulating which sub-types we use, the compiler can
allow more code to function. I think the reason for that is that function
boundaries act as strict guards for what's allowed, and as a rule are also
pretty coarse-grained. By making the more expressive, the compiler becomes less
strict, and in turn more code can be allowed. It really does feel like view
types and the like might represent a generational leap akin to NLL</a> in terms of
general language ergonomics. I think it's interesting to think about what else
could be loosened within the framework of Rust's existing semantics.</p>
Goals of this post</strong></p>
To re-iterate: this post is intentionally incomplete. I've knowingly glossed
over details, and left pieces out. The goal of the post was to show what the
outcomes</em> we could have if we combined some features folks are thinking about
anyway. Together with a personal goal to not spend months writing about this,
because I actually should be working on diffferent things. Rust's
roadmap is always a balance of team prioritization and personal interests; so
showing what can be achieved by combining several features might lower the bar
to make the case these features would be useful, or perhaps even get people
interested in working on this.</p>
Wrapping up</strong></p>
Anyway, that's probably enough discussion for now. This post was a massive
procrastination from writing another post on "halt points", which I've finished
drafting and am currently editing. This was is mostly thoughts which have been
stewing in the back of my mind after talking with colleages at work about state
machines, type systems, and more accurately modeling programs. Thanks for reading!</p>

Announcing the Keyword Generics Initiative
Wed, 27 Jul 2022 00:00:00 +0000

This post is a mirror of the post published to the Inside Rust
blog</a>.
If you want to read an earlier iteration of the ideas described in this post,
check out the post on
Async Overloading</a>,
written about a year earlier.</p>
</blockquote>

We (Oli</a>, Niko</a>, and Yosh</a>) are excited to announce the start of the Keyword
Generics Initiative</a>, a new initiative 1</a></sup> under the purview of
the language team. We're officially just a few weeks old now, and in this post
we want to briefly share why we've started this initiative, and share some
insight on what we're about.</p>
^{1</sup>
Rust governance terminology can sometimes get confusing. An
"initiative" in Rust parlance is different from a "working group" or "team".
Initiatives are intentionally limited: they exist to explore, design, and
implement specific pieces of work - and once that work comes to a close, the
initiative will wind back down. This is different from, say, the lang team -
which essentially carries a 'static</code> lifetime - and whose work does
not have a clearly defined end.</p>
</div>}A missing kind of generic</h2>
One of Rust's defining features is the ability to write functions which are
generic</em> over their input types. That allows us to write a function once,
leaving it up to the compiler to generate the right implementations for us.</p>
Rust allows you to be generic over types - it does not allow you to be generic
over other things that are usually specified by keywords. For example, whether a
function is async, whether a function can fail or not, whether a function is
const or not, etc.</p>
The post "What color is your function"</a> 2</a></sup> describes what happens
when a language introduces async functions, but with no way to be generic over
them:</p>

I will take async-await over bare callbacks or futures any day of the week.
But we’re lying to ourselves if we think all of our troubles are gone. As soon
as you start trying to write higher-order functions, or reuse code, you’re
right back to realizing color is still there, bleeding all over your codebase.</p>
</blockquote>
This isn't just limited to async though, it applies to all modifier keywords -
including ones we may define in the future. So we're looking to fill that gap
by exploring something we call "keyword generics" 3</a></sup>: the ability to be
generic over keywords such as const</code> and async</code>.</p>
^{2</sup>
R. Nystrom, “What Color is Your Function?,” Feb. 01, 2015.
https://journal.stuffwithstuff.com/2015/02/01/what-color-is-your-function/
(accessed Apr. 06, 2022).</p>
</div>}^{3</sup>
The longer, more specific name would be: "keyword modifier generics".
We've tried calling it that, but it's a bit of a mouthful. So we're just
sticking with "keyword generics" for now, even if the name for this feature may
end up being called something more specific in the reference and documentation.</p>
</div>
To give you a quick taste of what we're working on, this is roughly how we
imagine you may be able to write a function which is generic over "asyncness"
in the future:</p>}
Please note that this syntax is entirely made up, just so we can use something
in examples. Before we can work on syntax we need to finalize the semantics,
and we're not there yet. This means the syntax will likely be subject to
change over time.</p>
</blockquote>
async<A> </span>trait </span>Read {
</span>    async<A> </span>fn </span>read</span>(&</span>mut </span>self</span>, </span>buf</span>: &</span>mut</span> [</span>u8</span>]) -> Result<</span>usize</span>>;
</span>    async<A> </span>fn </span>read_to_string</span>(&</span>mut </span>self</span>, </span>buf</span>: &</span>mut</span> String) -> Result<</span>usize</span>> { ... }
</span>}
</span>
</span>/// Read from a reader into a string.
</span>async<A> </span>fn </span>read_to_string</span>(</span>reader</span>: &</span>mut</span> impl Read * A) -> std::io::Result<String> {
</span>    </span>let mut</span> string = String::new();
</span>    reader.</span>read_to_string</span>(&</span>mut</span> string).await?;
</span>    string
</span>}
</span></code></pre>
This function introduces a "keyword generic" parameter into the function of A</code>.
You can think of this as a flag which indicates whether the function is being
compiled in an async context or not. The parameter A</code> is forwarded to the impl Read</code>, making that conditional on "asyncness" as well.</p>
In the function body you can see a .await</code> call. Because the .await</code> keyword
marks cancellation sites</a> we unfortunately can't just infer them
4</a></sup>. Instead we require them to be written for when the code is
compiled in async mode, but are essentially reduced to a no-op in non-async
mode.</p>
^{4</sup>
No really, we can't just infer them - and it may not be as
simple as omitting all .await</code> calls either. The Async WG is working through
the full spectrum of cancellation sites, async drop, and more. But for now we're
working under the assumption that .await</code> will remain relevant going forward.
And even in the off chance that it isn't, fallibility has similar requirements
at the call site as async does.</p>
</div>
We still have lots of details left to figure out, but we hope this at least
shows the general feel</em> of what we're going for.</p>}A peek at the past: horrors before const</h2>
Rust didn't always have const fn</code> as part of the language. A long long long long
long time ago (2018) we had to write a regular function for runtime computations
and associated const of generic type logic for compile-time computations. As an
example, to add the number 1</code> to a constant provided to you, you had to write
(playground</a>):</p>
trait </span>Const<T> {
</span>    </span>const </span>VAL</span>: T;
</span>}
</span>
</span>/// `42` as a "const" (type) generic:
</span>struct </span>FourtyTwo;
</span>impl </span>Const<</span>i32</span>> </span>for </span>FourtyTwo {
</span>    </span>const </span>VAL</span>: </span>i32 </span>= </span>42</span>;
</span>}
</span>
</span>/// `C` -> `C + 1` operation:
</span>struct </span>AddOne<C: Const<</span>i32</span>>>(C);
</span>impl</span><C: Const<</span>i32</span>>> Const<</span>i32</span>> </span>for </span>AddOne<C> {
</span>    </span>const </span>VAL</span>: </span>i32 </span>= C::</span>VAL </span>+ </span>1</span>;
</span>}
</span>
</span>AddOne::<FourtyTwo>::</span>VAL
</span></code></pre>
Today this is as easy as writing a const fn</code>:</p>
const fn </span>add_one</span>(</span>i</span>: </span>i32</span>) -> </span>i32 </span>{
</span>    i + </span>1
</span>}
</span>
</span>add_one</span>(</span>42</span>)
</span></code></pre>
The interesting part here is that you can also just call this function in
runtime code, which means the implementation is shared between both const</code>
(CTFE5</a></sup>) and non-const</code> (runtime) contexts.</p>
^{5</sup>
CTFE stands for "Compile Time Function Execution": const</code> functions
can be evaluated during compilation, which is implemented using a Rust
interpreter (miri).</p>
</div>}Memories of the present: async today</h2>
People write duplicate code for async/non-async with the only difference being
the async</code> keyword. A good example of that code today is async-std</code></a>, which
duplicates and translates a large part of the stdlib's API surface to be async
6</a></sup>. And because the Async WG has made it an explicit goal to bring
async Rust up to par with non-async Rust</a>, the issue of code
duplication is particularly relevant for the Async WG as well. Nobody on the
Async WG seems particularly keen on proposing we add a second instance of just
about every API currently in the stdlib.</p>
^{6</sup>
Some limitations in async-std</code> apply: async Rust is missing async
Drop</code>, async traits, and async closures. So not all APIs could be duplicated.
Also async-std</code> explicitly didn't reimplement any of the collection APIs to be
async-aware, which means users are subject to the "sandwich problem". The
purpose of async-std</code> was to be a proving ground to test whether creating
an async mirror of the stdlib would be possible: and it's proven that it is, as
far as was possible with missing language features.</p>
</div>}We're in a similar situation with async</code> today as const</code> was prior to 2018.
Duplicating entire interfaces and wrapping them in block_on</code> calls is the
approach taken by e.g. the mongodb</code>
[async</a>,
non-async</a>], postgres</code>
[async</a>,
non-async</a>], and reqwest</code>
[async</a>,
non-async</a>] crates:</p>
// Async functionality like this would typically be exposed from a crate "foo":
</span>async </span>fn </span>bar</span>() -> Bar { 
</span>    </span>// async implementation goes here
</span>}
</span></code></pre>
// And a sync counterpart would typically be exposed from a crate
</span>// named "blocking_foo" or a submodule on the original crate as
</span>// "foo::blocking". This wraps the async code in a `block_on` call:
</span>fn </span>bar</span>() -> Bar {
</span>    futures::executor::block_on(foo::bar())
</span>}
</span></code></pre>
This situation is not ideal. Instead of using the host's synchronous syscalls,
we're now going through an async runtime to get the same results - something
which is often not zero-cost. But more importantly, it's rather hard to
keep both a sync and async API version of the same crate in, err, sync with each
other. Without automation it's really easy for the two APIs to get out of sync,
leading to mismatched functionality.</p>
The ecosystem has come up with some solutions to this, perhaps most notably the
proc-macro based maybe-async</code> crate</a>.  Instead of writing two
separate copies of foo</code>, it generates a sync and async variant for you:</p>
#[</span>maybe_async</span>]
</span>async </span>fn </span>foo</span>() -> Bar { ... }
</span></code></pre>
While being useful, the macro has clear limitations with respect to diagnostics
and ergonomics. That's absolutely not an issue with the crate, but an inherent
property of the problem it's trying to solve. Implementing a way to be generic
over the async</code> keyword is something which will affect the language in many
ways, and a type system + compiler will be better equipped to handle it than
proc macros reasonably can.</p>
A taste of trouble: the sandwich problem</h2>
A pervasive issue in existing Rust is the sandwich</em> problem. It occurs when a
type passed into an operation wants to perform control flow not supported by the
type it's passed into. Thus creating a sandwich</em> 7</a></sup> The classic example
is a map</code> operation:</p>
^{7</sup>
Not to be confused with the higher-order sandwich dilemma</em> which is
when you look at the sandwich problem and attempt to determine whether the
sandwich is two slices of bread with a topping in between, or two toppings with
a slice of bread in between. Imo the operation part of the problem feels more
bready</em>, but that would make for a weird-looking sandwich. Ergo: sandwich
dilemma. (yes, you can ignore all of this.)</p>
</div>
enum </span>Option<T> {
</span>    Some(T),
</span>    None,
</span>}
</span>
</span>impl</span><T> Option<T> {
</span>    </span>fn </span>map</span><J>(</span>self</span>, </span>f</span>: impl FnOnce(</span>T</span>) -> J) -> Option<J> { ... }
</span>}
</span>
</span>my_option.</span>map</span>(|</span>x</span>| x.await)
</span></code></pre>
This will produce a compiler error: the closure f</code> is not an async context, so
.await</code> cannot be used within it. And we can't just convert the closure to be
async</code> either, since fn map</code> doesn't know how to call async functions. In
order to solve this issue, we could provide a new async_map</code> method which
does</em> provide an async closure. But we may want to repeat those for more
effects, and that would result in a combinatorial explosion of effects. Take for
example "can fail" and "can be async":</p>
</th> not async</th> async</th></tr></thead>

infallible</strong></td> fn map</code></td> fn async_map</code></td></tr>
fallible</strong></td> fn try_map</code></td> fn async_try_map</code></td></tr>
</tbody></table>
That's a lot of API surface for just a single method, and that problem
multiplies across the entire API surface in the stdlib</strong>. We expect that once we
start applying "keyword generics" to traits, we will be able to solve the
sandwich problem. The type f</code> would be marked generic over a set of effects,
and the compiler would choose the right variant during compilation.</p>
Affecting all effects</h2>
Both const</code> and async</code> share a very similar issue, and we expect that other
"effects" will face the same issue. "fallibility" is particularly on our mind here,
but it isn't the only effect. In order for the language to feel consistent we
need consistent solutions.</p>
FAQ</h2>
Q: Is there an RFC available to read?</h3>
Rust initiatives are intended for exploration</em>. The announcement of the Keyword
Generics Initiative marks the start</em> of the exploration process. Part of
exploring is not knowing what the outcomes will be. Right now we're in the
"pre-RFC" phase of design. What we hope we'll achieve is to enumerate the
full problem space, design space, find a balanced solution, and eventually
summarize that in the form of an RFC. Then after the RFC is accepted: implement
it on nightly, work out the kinks, and eventually move to stabilize.  But we may
at any point during this process conclude that this initiative is actually
infeasible and start ramping down.</p>
But while we can't make any assurances</em> about the outcome of the initiative,
what we can share is that we're pretty optimistic about the initiative overall.
We wouldn't be investing the time we are on this if we didn't think we'd be
actually be able to see it through to completion.</p>
Q: Will this make the language more complicated?</h3>
The goal of keyword generics is not to minimize the complexity of the Rust
programming language, but to minimize the complexity of programming in Rust.</em>
These two might sound similar, but they're not. Our reasoning here is that by
adding</em> a feature, we will actually be able to significantly reduce the surface
area of the stdlib, crates.io libraries, and user code - leading to a more
streamlined user experience.</p>
Choosing between sync or async code is a fundamental choice which needs to be
made. This is complexity which cannot be avoided, and which needs to exist
somewhere. Currently in Rust that complexity is thrust entirely on users of
Rust, making them responsible for choosing whether their code should support
async Rust or not. But other languages have made diferent choices. For example
Go doesn't distinguish between "sync" and "async" code, and has a runtime which
is able to remove that distinction.</p>
In today's Rust application authors choose whether their application will be sync
or async, and even after the introduction of keyword generics we don't really
expect that to change. All generics eventually need to have their types known,
and keyword generics are no different. What we're targeting is the choice made
by library</em> authors whether their library supports is sync or async. With
keyword generics library authors will be able to support both with the help of
the compiler, and leave it up to application authors to decide how they want to
compile their code.</p>
Q: Are you building an effect system?</h3>
The short answer is: kind of, but not really. "Effect systems" or "algebraic
effect systems" generally have a lot of surface area. A common example of what
effects allow you to do is implement your own try/catch</code> mechanism. What we're
working on is intentionally limited to built-in keywords only, and wouldn't
allow you to implement anything like that at all.</p>
What we do share with effect systems is that we're integrating modifier keywords
more directly into the type system. Modifier keywords like async</code> are often
referred to as "effects", so being able to be conditional over them in
composable ways effectively gives us an "effect algebra". But that's very
different from "generalized effect systems" in other languages.</p>
Q: Are you looking at other keywords beyond async</code> and const</code>?</h3>
For a while we were referring to the initiative as "modifier generics" or
"modifier keyword generics", but it never really stuck. We're only really
interested in keywords which modify how types work. Right now this is const</code>
and async</code> because that's what's most relevant for the const-generics WG and
async WG. But we're designing the feature with other keywords in mind as well.</p>
The one most at the top of our mind is a future keyword for fallibility. There
is talk about introducing try fn() {}</code> or fn () -> throws</code> syntax. This could
make it so methods such as Iterator::filter</code> would be able to use ?</code> to break
out of the closure and short-circuit iteration.</p>
Our main motiviation for this feature is that without it, it's easy for Rust to
start to feel disjointed</em>. We sometimes joke that Rust is actually 3-5
languages in a trenchcoat. Between const rust, fallible rust, async rust, unsafe
rust - it can be easy for common APIs to only be available in one variant of the
language, but not in others. We hope that with this feature we can start to
systematically close those gaps, leading to a more consistent Rust experience
for all</em> Rust users.</p>
Q: What will the backwards compatibility story be like?</h3>
Rust has pretty strict backwards-compatibility guarantees, and any feature we
implement needs to adhere to this. Luckily we have some wiggle room because of
the edition mechanism, but our goal is to shoot for maximal backwards compat. We
have some ideas of how we're going to make this work though, and we're
cautiously optimistic we might actually be able to pull this off.</p>
But to be frank: this is by far one of the hardest aspects of this feature, and
we're lucky that we're not designing any of this just by ourselves, but have the
support of the language team as well.</p>
Q: Aren't implementations sometimes fundamentally different?</h3>
Const Rust can't make any assumptions about the host it runs on, so it can't do
anything platform-specific. This includes using more efficient instructions of
system calls which are only available in one platform but not another. In order
to work around this, the const_eval_select</code></a> intrinsic in the standard
library enables const</code> code to detect whether it's executing during CTFE or
runtime, and execute different code based on that.</p>
For async we expect to be able to add a similar intrinsic, allowing library
authors to detect whether code is being compiled as sync or async, and do
something different based on that. This includes: using internal concurrency, or
switching to a different set of system calls. We're not sure whether an
intrinsic is the right choice for this though; we may want to provide a more
ergonomic API for this instead. But because keyword generics is being designed
as a consistent feature, we expect that whatever we end up going with can be used
consistently by all</em> modifier keywords.</p>
Conclusion</h2>
In this post we've introduced the new keyword generics initiatve, explained why
it exists, and shown a brief example of what it might look like in the future.</p>
The initiative is active on the Rust Zulip under
t-lang/keyword-generics</code></a> - if this seems interesting to you, please
pop by!</p>
Thanks to everyone who's helped review this post, but in particular:
fee1-dead</a>, Daniel Henry-Mantilla</a>, and Ryan Levick</a></em></p>

Keywords I: Unsafe Syntax
Sat, 09 Jul 2022 00:00:00 +0000
Last week Oli</a> and I were thinking through some examples involving modifiers and
impl</code> blocks. Namely: would it be possible or even useful to declare something
like this:</p>
// all methods in this block are `const`.
</span>const impl </span>MyStruct {
</span>    </span>fn </span>foo</span>(&</span>mut </span>self</span>) {}
</span>    </span>fn </span>bar</span>(&</span>mut </span>self</span>) {}
</span>}
</span></code></pre>
We thought this looked alright. But something we realized is that if we allowed
this for const</code>, we'd likely want to allow this for all modifier keywords too,
including unsafe</code>. And that unfortunately causes some issues. So this post is a
brief look at what those issues are, what plans exist to improve it, and how we
might even be able to do things better?</p>
Note: The purpose of this post is mostly to share thoughts. I don't speak for
Oli here, and I definitely don't speak for any Rust team. Don't take think of
this as a serious proposal, but instead think of it as notes which may come in
useful at a later time.</em></p>
Two meanings of unsafe</h2>
unsafe</code></a> in Rust is actually two keywords in a trenchcoat. Depending on where
you use it, it'll mean something different:</p>

On definition: unsafe fn</code> / unsafe trait</code></li>
On usage: unsafe {}</code> / unsafe impl</code></li>
</ul>
Each provides the following contract:</p>

On definition: there are additional invariants which the compiler cannot
check, so the programmer must check those by hand instead.</li>
On usage: I'm manually checking invariants here which the compiler cannot
check for me.</li>
</ul>
Together these form a pair - not unlike async/.await</code>. With the slight
difference that defining an unsafe</code> block in a function body does not come with
the additional restriction that the function itself must also be marked as
unsafe</code>:</p>
 </span>// using `.await` requires you wrap it in an `async` context, which propagates
</span> </span>// to the next caller:
</span>async </span>fn </span>foo</span>() {
</span>    </span>bar</span>().await;
</span>}
</span>
</span>// however using `unsafe` doesn't require you wrap it in an `unsafe` context;
</span>// `fn foo` doesn't also need to be `unsafe fn foo` here:
</span>fn </span>foo</span>() {
</span>    </span>unsafe </span>{ </span>bar</span>() };
</span>}
</span></code></pre>
And this makes sense: pretty much all of Rust builds on some foundation of
unsafe</code>. So if unsafe</code> was transitive much the same way async/.await</code> is,
then virtually every function would need to be marked unsafe</code> - which would defeat the point of using unsafe</code> as a sign post for where we're manually checking invariants.</p>
Bringing it back to traits</h2>
So okay, that's all well and good. But I mentioned we got onto this because we were
exploring the idea of allow keyword modifiers on entire blocks. Which is a
useful thing to have if, idk, you're in the process of marking the majority of
the stdlib as const</code> 1</a></sup>. If you're able to declare things in a more
general way it makes things, uh, flow a bit more.</p>
^{1</sup>
In practice very few APIs cannot ever be called from const fn</code> functions. Those are: all APIs which need to have knowledge of the host system (OS, filesystem, network devices, etc.) or have access to statics. I think that's like, less than 10% of the total API surface in the stdlib; meaning the vast majority may eventually be constified. Not even to being about adding other types of effects (cough</em> async cough</em>) to the stdlib as well.</p>
</div>
But here's where it gets odd. Take the following code:</p>}struct </span>MyStruct;
</span>
</span>unsafe impl </span>Send </span>for </span>MyStruct {}
</span></code></pre>
This says: "we promise the implementation of Send</code> here is safe". But if we
adapt our const impl</code> example from earlier to be unsafe</code> impl instead we get
this:</p>
// The intended meaning here is:
</span>// "all methods in this block are `unsafe` to call"
</span>unsafe impl </span>Foo {
</span>    </span>fn </span>foo</span>(&</span>mut </span>self</span>) {}
</span>    </span>fn </span>bar</span>(&</span>mut </span>self</span>) {}
</span>}
</span></code></pre>
This has the exact opposite meaning of our earlier unsafe impl</code>! Doing this
means we'd have a semantic difference between inherent impls 2</a></sup> and
trait impls. Which is a suuuuuper subtle distinction, and one which is almost guaranteed to trip people up!</p>
^{2</sup>
This is jargon for: "A regular impl block on a type". So not the
trait impl block.</p>
</div>}Do we have options?</h2>
There's some things we might be able to do here. First option:</p>

Do nothing!</p>
</blockquote>
Yes, we could definitely choose to not care about this and move on. But that's
an intellectual dead end, and I don't like those. I'm choosing to care about
this instead. At least enough to write about it and think about it and think things through a bit more.</p>

Color within the lines!</p>
</blockquote>
So what if we choose to restrict ourselves to making only additive changes here.
No breaking changes allowed. Well, we could yield that unsafe impl</code> has the
meaning it has, and we can't change that.  So maybe we could for example change
where we put the keyword to have different meanings. What if we
could make it something like this instead:</p>
struct </span>MyStruct;
</span>
</span>// `unsafe impl` - this impl has been checked
</span>unsafe impl </span>Send </span>for </span>MyStruct {}
</span>
</span>// `impl unsafe` - the methods here need to be checked when used
</span>impl </span>unsafe MyStruct {
</span>    </span>fn </span>foo</span>(&</span>mut </span>self</span>) {}
</span>    </span>fn </span>bar</span>(&</span>mut </span>self</span>) {}
</span>}
</span></code></pre>
We could make it so unsafe impl</code> means: "this impl has been checked". And impl unsafe</code> means: "this implementation needs to be checked. That's still subtle,
but at least it would no longer be identical. Adapting it to const</code> would then
be:</p>
// `const impl` would be disallowed; that's for `unsafe` only.
</span>impl </span>const MyStruct {
</span>    </span>fn </span>foo</span>(&</span>mut </span>self</span>) {}
</span>    </span>fn </span>bar</span>(&</span>mut </span>self</span>) {}
</span>}
</span></code></pre>
This is workable, but imo not ideal. When I say things out loud I definitely say
it's a "const impl" or "async impl". But we'd be writing it the other way
around.</p>
Okay, so what if we start coloring outside the lines?</h2>
What if we actually took a step back and asked ourselves: "if we
could start from scratch, what would we change about unsafe</code>?" Well, for
starters, I think our memory
model</a> is looking pretty
good! So we wouldn't want to bother too much there, and focus more on the
syntax.</p>
Today having an unsafe fn</code> implies the function body to be an
unsafe {}</code> block. RFC
2585 (2020)</a>
was merged to break that assumption; presumably over the course of a few
editions (though afaict no decisions have been made on the exact process). But
that would move us one step closer to differentiating between the two meanings
of unsafe</code>.</p>
But what if we deprecated unsafe</code> entirely instead? RFC
117 (2014)</a> suggested we do exactly this,
albeit pre-1.0. But given the semantics</em> of unsafe</code> wouldn't change, but only
the syntax would, this is the type of change I believe we could run over an
edition bound.</p>
Personally I like checked</code> / unchecked</code> to differentate between the two types
of unsafe</code> 3</a></sup>. It's common
terminology</a> in the stdlib today
already, and it would solve the problem of "what did you mean by unsafe</code>"
entirely. Our code examples could then be written like this:</p>
^3</sup>I like "checked" more than I like "safe". When I hear safe</em> to me it
sounds like we've promised there are no bugs. And to quote Dijkstra: Testing
shows the presence, not the absence of
bugs</a>. At best</em>
we can guarantee we've checked for bugs. And its up to others to decide whether
they deem that safe enough.</p>
</div>
struct </span>MyStruct;
</span>
</span>// this implementation has been checked
</span>checked </span>impl </span>Send </span>for </span>MyStruct {}
</span>
</span>// these methods need to be checked
</span>unchecked </span>impl </span>MyStruct {
</span>    </span>fn </span>foo</span>(&</span>mut </span>self</span>) {}
</span>    </span>fn </span>bar</span>(&</span>mut </span>self</span>) {}
</span>}
</span></code></pre>
See! No more ambiguity. And I think that'd be a huge step up from the status
quo! To spell out the other uses as well, here's all existing uses of unsafe</code>
translated to use "check" terminology:</p>
// `unsafe fn`
</span>unchecked </span>fn </span>foo</span>() {}
</span>
</span>// `unsafe trait`
</span>unchecked </span>trait </span>MyTrait {}
</span>
</span>// `unsafe impl`
</span>checked </span>impl </span>Send </span>for </span>MyStruct {}
</span>
</span>// `unsafe {}`
</span>fn </span>foo</span>() {
</span>    checked { </span>bar</span>() };
</span>}
</span></code></pre>
But owo there's more!</h2>
What? There's more?</p>
Yes! Because we now have a keyword pair, we can take a page out of the book of
other keyword pairs. Ergonomics matter in all parts of a language, but
especially so in places where the stakes are high. So what if we allowed</p>
In RFC 2585 Niko</a> showed that many unsafe</code> functions today are one-liners, and
making it so unsafe fn</code> does not imply unsafe {}</code> would yield the following
result:</p>
unsafe fn </span>foo</span>(...) -> ... { </span>unsafe </span>{
</span>  </span>// Code goes here
</span>} }
</span></code></pre>
Now, what if... we did away with unsafe {}</code> blocks (or at least supplemented
them) and added a postfix operator operator? What if we could do this instead!:</p>
unchecked </span>fn </span>foo</span>(...) -> ... { 
</span>    </span>bar</span>().checked
</span>}
</span></code></pre>
We can probably debate exact words and syntax all day long. But the idea doesn't
seem too bad, does it? Now granted, perhaps there are solid reasons to keep
blocks as well. But it's common advice that unsafe</code> blocks should be scoped as
tightly as possible. In today's rust that often means we're writing things like:</p>
unsafe fn </span>foo</span>(...) -> ... { 
</span>    </span>unsafe </span>{ </span>bar</span>() }.</span>baz</span>()
</span>}
</span></code></pre>
And I think in comparison having a postfix notation would look a lot better:</p>
unchecked </span>fn </span>foo</span>(...) -> ... { 
</span>    </span>bar</span>().checked.</span>baz</span>()
</span>}
</span></code></pre>
Going even further?</h2>
The core principle of Rust is that it can wrap unchecked internals into a
package which exposes checked semantics. This allows us to clearly scope which
parts of the program need to deal with manually checking invariants, and allows
the rest of the program to not have to care about any of that. Instead the
compiler helps us.</p>
But not all checks are equal. I trust the stdlib devs to have carefully poured
over every single line of unsafe</code> and done their best to ensure it's correct.
At worst there might be a bug. But when I'm downloading code from third parties,
well, we can be exposed to far more than bugs</a> 4</a></sup>.</p>
^{4</sup>
I believe this is the first supply chain attack we've become aware
of. It does something bad during compilation. But a malicious crate might just
as well choose to do something bad during runtime</em>. Supply chain attacks are
real, and as Rust grows, so will the frequency of attacks.</p>
</div>
unsafe</code> is the building block that everything else is built on. Allowing all
code we download to do anything at all is not great. I mean: it's great</em>
insofar that we can wrap openssl</code>, libgit</code>, winapi</code>, and more. But we tend to vet those packages, and make sure they're from trusted sources. It's not
great when that one random protocol parser I downloaded was actually turned out to be a piece of ransomware.</p>}Sunfish</a> wrote a thought
experiment</a> about what
it would take to create a sandbox backed by the Rust type system. And at least
we'd need to have a knob to turn off giving unrestricted access to unsafe</code> to every function in every dependency we have.</p>
pub fn </span>im_a_sweet_and_innocent_parser_uwu</span>(</span>reader</span>: impl Read) -> ParsedTree {
</span>    </span>haha_no_im_not_lemme_break_into_ur_unpatched_kernel_real_quick</span>();
</span>    ParsedTree::new(reader)
</span>}
</span></code></pre>
The reason why I'm raising it here is because in a way this goes in the
opposite</em> direction of RFC 2585. Instead of decoupling unsafe fn</code> from unsafe {}</code> it almost creates a tighter relationship between the two - but through
other means. In order to use unsafe {}</code> / checked {}</code> anywhere in a function,
the function signature would need to reflect that.</p>
We wouldn't want to make the unsafe</code> keyword transitive so this would need to
behave differently than async/.await</code>. But in order to enable</em> unsafe</code> to be
used, we'd want to share the capability</em> to do so via the function signature.
The only types excluded from this mechanism would be the stdlib; since if you
trust the compiler you should also trust the stdlib.</p>
// `foo` needs maximum permissions ("ambient authority") to
</span>// perform `unsafe`/`checked` operations:
</span>fn </span>foo</span>()
</span>with
</span>    ambient_authority
</span>{
</span>    </span>bar</span>().checked
</span>}
</span></code></pre>
This would allow us to start flagging which functions depend on having the keys
to the castle, and which do not. Obviously that's not the full story,
capabilities and especially capability safety</em> are a pretty deep topic in and
of themselves. But because they would affect the function signature of unsafe</code> types, it's worth bringing up here.</p>
Conclusion</h2>
Okay, so we kind of went hard on the changes we could plausibly want to make to
unsafe</code> in the future. But I think that's fine! - We at least figured a
plausible way out of the main drawback of RFC 2585 - though I guess a postfix
unsafe</code> keyword could hold water even without the rename to
checked/unchecked</code>.</p>
But equally importantly this would actually open the path to allowing
const impl {}</code>, async impl</code>, and possibly other effects in the future as well.
And that's probably needed, because as we're looking to make our typesystem more
powerful, we don't want people to have to actually type</em> more (get it) to make
use of it.</p>
This initially was a thread on
twitter</a>, but I
figured it might be worth giving it some permanence. I hope you enjoyed it; love
you.</p>

Async Cancellation II: Time and Signals
Fri, 10 Jun 2022 00:00:00 +0000
For the past few years I've been working on the async-std</code> library, which
provides an async implementation of the APIs exposed by std</code>. However, we also
added several new APIs related to things unique to async Rust: concurrency,
control over execution, and the interaction between the two.</p>
These APIs were initially introduced in async-std</code> as "unstable", and have been
the main focus of my work to design since. On this blog there are numerous posts
related to for example:
concurrency</a>,
cancellation</a>, and
parallelism</a>. Today I want to
share a new experiment I've been working for time-based operations in async
Rust. I've designed it as a stand-alone crate for now, but I intend to PR its
addition to async-std</code> in the near future.</p>
Timer basics</h2>
Relevant APIs here are:</em></p>

futures_time::task::Sleep</code></a></li>
futures_time::stream::Interval</code></a></li>
</ul>
In order to do anything with time you need to have timers. There are two basic
types of timers: a timer which fires once at a specific time, and a timer which
fires repeatedly after each interval 1</a></sup>. In futures-time</code> you can create both
these timers by concstructing these futures directly. But more interestingly: we
allow people to construct these futures by convert an existing</em>
time::Duration</code> or time::Instant</code> directly into a timer through the
IntoFuture</code> and IntoStream</code> traits.</p>
^{1</sup>
People would be right to point out that Interval</code> can be
implemented in terms of Sleep</code>. But even though that's the case, APIs will have
to choose between "do I fire once" or "do I fire repeatedly", and for that
reason it's worth distinguishing between the two. Besides: Interval</code> is so common
that we don't want folks to have to implement it over and over, so we just provide it
as one of the two basic timer types.</p>
</div>
Because both these traits are also implemented for all T: Future</code> and
T: Stream</code> respectively, time-based APIs can take timer: impl IntoFuture</code> or
instant: impl IntoStream</code> and pass either Duration</code>/Instant</code> directly, or
another future.</p>}use </span>futures_timer::prelude::*;
</span>use </span>futures_timer::{task, time::Duration};
</span>
</span>async </span>fn </span>wait_until</span>(</span>timer</span>: impl IntoFuture) {
</span>    </span>let</span> timer = timer.</span>into_future</span>();
</span>    timer.await;
</span>}
</span>
</span>#[</span>async_std</span>::</span>main</span>]
</span>async </span>fn </span>main</span>() {
</span>    </span>// Wait for a `Duration`.
</span>    </span>wait_until</span>(Duration::from_secs(</span>1</span>));
</span>
</span>    </span>// Wait for a concrete future.
</span>    </span>let</span> deadline = task::sleep(Duration::from_secs(</span>1</span>));
</span>    </span>wait_until</span>(deadline);
</span>}
</span></code></pre>
To simplify working with the orphan rules, we ended up defining our own variants
of IntoFuture</code>, IntoStream</code>, Duration</code>, and Instant</code>. When we move this to
async-std</code> we can probably simplify some of it - but we'll likely need to keep
our own definitions of Duration</code> and Instant</code> just for this to keep working.</p>
Cancelling execution at a distance</h2>
Relevant APIs here are:</em></p>

futures_time::future::FutureExt::timeout</code></a></li>
futures_time::stream::StreamExt::timeout</code></a></li>
</ul>
As I showed in the last section, thanks to the implementation of the Into*</code>
traits functions can be agnostic over whether they take a Duration</code>/Instant</code>
instance, or whether they take a Future</code>. For the last three years we've been
trying to figure out how we could integrate the stop-token</code>
crate</a> into async-std</code>.</p>
For a deep-dive into how cancellation works in async Rust, you can read the
last post in this series</a>.
But the short version is: futures can be cancelled between .await</code> points just
by dropping the future. A timeout is nothing more than a "race" between a future
F</code> and a timer future. If F</code> completes first, you get an Ok(value)</code>. And if the
timer completes first, you get an Err(TimeoutError)</code>. That's concepts composing
nicely with each other!</p>
We tried various APIs for cancellation, but none really felt good. So I figured I
just needed to give it some time, and started looking at async temporal
operations. When I saw RxJS's
timeout</code></a> and
timeoutWith</code></a> operators I realized
we might actually be able to combine both functionality through a single API. And instead of using a dedicated
StopSource</code></a> /
StopToken</code></a>
pair, we can instead reuse the async-channel</code>
crate</a> to create a sender + receiver.
This is what it looks like when used together:</p>
use </span>futures_lite::prelude::*;
</span>use </span>futures_time::prelude::*;
</span>use </span>futures_time::channel;
</span>use </span>futures_time::time::Duration;
</span>
</span>fn </span>main</span>() {
</span>    async_io::block_on(async {
</span>        </span>let </span>(send, </span>mut</span> recv) = channel::bounded::<()>(</span>1</span>); </span>// create a new send/receive pair
</span>        </span>let</span> value = async { "</span>meow</span>" }
</span>            .</span>delay</span>(Duration::from_millis(</span>100</span>))
</span>            .</span>timeout</span>(recv.</span>next</span>()) </span>// time-out when the sender emits a message
</span>            .await;
</span>
</span>        assert_eq!(value.</span>unwrap</span>(), "</span>meow</span>");
</span>    })
</span>}
</span></code></pre>
This example will always return meow</em> because send</code> is never dropped. But in a
concurrent example, a timeout could be triggered for all instances of recv</code>
(which implements Clone</code>) by dropping send</code>. This is pretty nice because it
combines several individually useful components to create "cancel at a distance
semantics" which are usually reserved for dedicated APIs.</p>
But there are still several rough edges in this API, which I think we may be
able to address:</p>

There are multiple imports from different crates required to get this to
work. Though this will likely be smoothed out somewhat by exposing it from
async-std</code>.</li>
When constructing a channel you have to declare a mut recv</code> instead of
recv</code>. This is because recv.next()</code> expects &mut self</code>. Implementing
IntoFuture</code> for async_channel::Receiver</code> would remove that requirement,
streamlining things somewhat.</li>
channel::bounded</code> needs to know what type it is, so it takes a ::<()></code>
turbo-fish. Creating zero-sized channels for messaging is useful on its own,
so once we're able to </a> we
may want the constructor to default to a zero-sized type.</li>
</ol>
If we migrate to async-std</code> and have all these changes applied, usage might end
up looking somewhat like this:</p>
use </span>async_std::prelude::*;
</span>use </span>async_std::channel;
</span>use </span>async_std::time::Duration;
</span>
</span>#[</span>async_std</span>::</span>main</span>]
</span>async </span>fn </span>main</span>() {
</span>    </span>let </span>(send, recv) = channel::bounded(</span>1</span>);
</span>    </span>let</span> value = async { "</span>meow</span>" }
</span>        .</span>delay</span>(Duration::from_millis(</span>100</span>))
</span>        .</span>timeout</span>(recv)
</span>        .await;
</span>
</span>    assert_eq!(value.</span>unwrap</span>(), "</span>meow</span>");
</span>}
</span></code></pre>
Delaying execution</h2>
Relevant APIs here are:</em></p>

futures_time::future::FutureExt::delay</code></a></li>
futures_time::stream::StreamExt::delay</code></a></li>
futures_time::future::FutureExt::park</code></a></li>
futures_time::stream::StreamExt::park</code></a></li>
</ul>
Arbitrary cancellation of execution might be one of the most important features of async Rust,
but it's not the only one. By gaining control over execution we also gain the
ability to delay, pause, and resume execution as well. Here's an example of how
to delay an execution by a set time limit, or until a signal is sent:</p>
use </span>futures_time::prelude::*;
</span>use </span>futures_time::time::{Instant, Duration};
</span>use </span>std::future;
</span>
</span>fn </span>main</span>() {
</span>    async_io::block_on(async {
</span>        </span>let</span> value = future::ready("</span>meow</span>")
</span>            .</span>delay</span>(Duration::from_millis(</span>100</span>)) </span>// delay resolving the future by 100ms
</span>            .await;
</span>        assert_eq!(value, Some("</span>meow</span>"));
</span>    });
</span>}
</span></code></pre>
I just authored the park</code> methods for this post, but didn't get around
to writing examples for them yet. park</code> works similar to std::thread::park</code></a>,
but instead of only working on threads it works with any</em> future or stream. You
call it by passing it a channel which can park or unpark the future, allowing
for convenient pausing and resuming of execution.</p>
In a previous post</a> I
mentioned that I suspect the right analogy for "task" is "parallelizable future"</em>
rather than "async thread"</em>, and showing that we can implement park</code> for all</em>
futures rather than only on Task</code> is another indication that this new
model may in fact be the right one.</p>
Throttling execution</h2>
Relevant APIs here are:</em></p>

futures_time::stream::StreamExt::buffer</code></a></li>
futures_time::stream::StreamExt::debounce</code></a></li>
futures_time::stream::StreamExt::sample</code></a></li>
futures_time::stream::StreamExt::throttle</code></a></li>
</ul>
Generally data loss is considered a bad thing. But sometimes… data loss can be a
good thing? Take for example mouse presses: when something is clicked once, you
want to handle that. But if something is pressed 15 times in .3 seconds, perhaps
not so much. This is an example of "external input", and since we cannot apply
backpressure to external conditions, the only method we have to reduce strain on
our system is to actively drop load.</p>
In the crate we provide some versatile, basic methods to shed load based on
different circumstances. The debounce</code> and sample</code> methods get the last item
seen during a particular period. The throttle</code> method will let through the first items received
during a particular period. And finally buffer</code> groups data from a particular
time range into a vector, allowing operators to operate on chunks of data rather
than needing to decide on individual items.</p>
use </span>futures_lite::prelude::*;
</span>use </span>futures_time::prelude::*;
</span>use </span>futures_time::time::{Instant, Duration};
</span>use </span>futures_time::stream;
</span>
</span>fn </span>main</span>() {
</span>    async_io::block_on(async {
</span>        </span>let mut</span> counter = </span>0</span>;
</span>        stream::interval(Duration::from_millis(</span>5</span>)) </span>// yield every 5ms
</span>            .</span>take</span>(</span>10</span>)                              </span>// stop after 10 iterations
</span>            .</span>buffer</span>(Duration::from_millis(</span>20</span>))     </span>// create 20ms windows
</span>            .</span>for_each</span>(|</span>buf</span>| counter += buf.</span>len</span>())  </span>// sum all items
</span>            .await;
</span>
</span>        assert_eq!(counter, </span>10</span>);
</span>    })
</span>}
</span></code></pre>
Conclusion</h2>
And that concludes the whirlwind-tour of the
futures-time</code></a> crate. I'm pretty happy with the
resulting APIs. For the longest time we were struggling to create APIs which
could operate on {Duration, Instant, impl Future}</code>, and this usually resulted in
needing to declare three different APIs for each piece of functionality. The
insight that we could implement IntoFuture</code> for Instant</code> and Duration</code> closed
that gap:</p>
async </span>fn </span>run</span>(</span>mut </span>recv</span>: Receiver<()>) {
</span>                                                                </span>// Notice how we're using the same API:
</span>    </span>let</span> db = </span>start_db</span>().</span>timeout</span>(Duration::from_secs(</span>10</span>)).await; </span>// Timeout based on a duration
</span>    </span>let</span> server = </span>start_server</span>().</span>timeout</span>(recv.</span>next</span>()).await;     </span>// Timeout based on a signal
</span>}
</span></code></pre>
One of the things I'm least sure about though is what it feels like to convert a
Duration</code> to a Future</code> outside a trait bound. Admittedly this feels somewhat
awkward to me:</p>
use </span>futures_time::time::Duration;
</span>use </span>futures_time::prelude::*;
</span>
</span>async </span>fn </span>main</span>() {
</span>    Duration::from_secs(</span>2</span>).await; </span>// await a duration of 2 secs?
</span>}
</span></code></pre>
I'm not sure. It might be the verb-noun thing, but it feels a bit weird. But
remembering an API Sean McArthur proposed years ago, we might be able to
streamline this a bit using conversion traits2</a></sup>:</p>
^2</sup>Note that IntoDuration</code> does not yet exist, but here's a playground link</a> to what that might look like.</p>
</div>
use </span>futures_time::time::Duration;
</span>use </span>futures_time::prelude::*;
</span>use </span>into_duration::IntoDuration;
</span>
</span>async </span>fn </span>main</span>() {
</span>    </span>2.</span>secs</span>().await; </span>// wait for 2 secs.
</span>}
</span></code></pre>
This reads a bit more fluent than what we might be used from most Rust APIs, but
I don't think that's a bad thing. I think it just takes some getting used to.
And it actually works quite nicely in other places too:</p>
use </span>async_std::prelude::*;
</span>use </span>async_std::channel;
</span>use </span>async_std::time::Duration;
</span>
</span>#[</span>async_std</span>::</span>main</span>]
</span>async </span>fn </span>main</span>() {
</span>    </span>let</span> value = async { "</span>meow</span>" }
</span>        .</span>delay</span>(</span>100.</span>millis</span>())     </span>// <- using this API
</span>        .await;
</span>
</span>    assert_eq!(value.</span>unwrap</span>(), "</span>meow</span>");
</span>}
</span></code></pre>
And thinking further ahead: if we get async IntoIterator</code> + async iteration
syntax, having it implemented for Duration</code> could allow us to write this:</p>
let</span> start = Instant::now();
</span>for</span> await now in </span>1.</span>secs</span>() {          </span>// loop every 1 second
</span>    dbg!(now.</span>duration_since</span>(start)); </span>// print elapsed time
</span>}
</span></code></pre>
Either way, there are some fun options available here. Implementing IntoFuture</code>
/ async IntoIterator</code> for Duration</code> is one of the things I'm still least sure
about - but I wonder how much of it is just a matter of getting used to
something novel that's now suddenly possible. I really don't know.</p>
That said though; I'm really happy with the way the library has turned out. And
I hope with some feedback and iteration it'll be good enough to include into
async-std</code>, and eventually also help us inform whether and how we want to expose this
functionality from the stdlib.</p>
There are a bunch of time-based APIs we haven't covered. But we certainly have
covered the fundamentals. And in particular I'm happy that we've finally found a
promising API for remote cancellation / pausing / resumption of arbitrary
futures. This is one of the key features async Rust provides, and actually
providing APIs for them seems important.</p>
Either way, that's what I have for now. If you want to check out the repo, you
can find it here:
yoshuawuyts/futures-time</code></a>. If
you'd like to keep up with my thoughts in near-real-time you can follow me on
Twitter</a>. And if you think the work I'm doing
is useful and you'd like to send me a tip, you can sponsor me
here</a>. Thanks!</p>

Postfix Spawn
Fri, 04 Mar 2022 00:00:00 +0000
Using a free spawn</code> function to create new tasks has been a staple</a> in async
Rust for many years. When we introduced async-std</code></a> in 2019 we took this
further by not only providing a spawn</code> function, but fully emulating the
std::thread</code></a> family of APIs and returning. But just because something has been around for a
while doesn't mean it's a good idea. And we know we have issues. In this post I
want to cover some of those issues, and share an experiment I've been working on
which may help with this.</p>
Structured Concurrency</h2>
In my previous post I covered the idea of structured concurrency</a>.
Though it's worth expanding on this concept more  in a separate post
1</a></sup>, here's the definition we'll be using for the sake of this post:</p>

Child tasks should never outlive their parent tasks (this includes destructors).</li>
If we cancel a parent task the child task should be cancelled too.</li>
If a child task raises an error, the parent task should be able to catch it.</li>
</ol>
The benefit of this is that code like this will never have dangling tasks</em>, and
will largely run in a predictable manner. This has all sorts of interesting
properties, such as graceful shutdown trivially falling out of this model - but
as I mentioned, that's something for a different post. The takeaway should be that
"structured concurrency" is a desirable property for any concurrent code to have.</p>
^{1</sup>
I've been meaning to for a while now, but alas. Time limits are a thing.</p>
</div>}Dangling by default</h2>
As we mentioned in the same post last time, by default most popular runtimes do
not behave in a manner which is compatible with structured concurrency. When a
JoinHandle</code> is dropped it isn't joined or cancelled, instead it's
detached</em> causing it to dangle.</p>
This behavior matches that of std::thread::JoinHandle</code> as well, but as Matklad
remarked years ago</a> that behavior in threads is not without issues. In hindsight "join on
drop" would have been a better default, but we can't easily change the behavior
of std::thread</code> anymore. What we can</em> do is ensure that the behavior of asynchronous tasks
default to the right thing. After all: modeling tasks after threads was a
choice</em>, we can just as easily make a different choice. std::task</code> is also the
one API we already don't expect to want to overload</a>, which means
we're not as restricted in the API space as we are with other async stdlib APIs.</p>
So if threads are not the right type to emulate the behavior of for tasks, what
might be? I suspect the answer is Future</code>. The futures API RFC</a>
mentions that cancelling futures should be done by dropping them.
Which means that if our tasks were to emulate futures, we should have
"cancel-on-drop" semantics too - as opposed to "join on drop" semantics.</p>
There already exists a runtime which implements this in the wild: smol</code></a>, which
is powered by the async-task</code></a> crate. "cancel-on-drop" semantics</a> mean that
cancellation of an outer task will automatically propagate to inner tasks as
well. Making it structurally concurrent by default.</p>
async let</h2>
"What if the behavior of futures and tasks were closer to each other"</em> has
been an intrusive thought for me lately. Creating futures in Rust is
easy, but spawning tasks is significantly harder. Why is that? It turns that the
people working on Swift were pondering something similar not too long ago, and
they've implemented a solution: async let</code></a>. Here's a
basic example of async let</code> in action:</p>
// given: 
</span>//   func chopVegetables() async throws -> [Vegetables]
</span>//   func marinateMeat() async -> Meat
</span>//   func preheatOven(temperature: Int) async -> Oven
</span>
</span>func </span>makeDinner() async throws -> Meal {
</span>  async </span>let</span> veggies = chopVegetables()
</span>  async </span>let</span> meat = marinateMeat()
</span>  async </span>let</span> oven = preheatOven(temperature: </span>350</span>)
</span>
</span>  </span>let</span> dish = Dish(ingredients: await [</span>try</span> veggies, meat])
</span>  </span>return try</span> await oven.cook(dish, duration: .hours(</span>3</span>))
</span>}
</span></code></pre>
Each instance of async let</code> creates a Swift type equivalent to our Task</code> type in Rust.
And awaiting an array of them is akin to our future::join</code> operations.
Swift provides a global runtime out of the box, and awaiting one of these tasks
schedules it on the runtime, and executes it in parallel. This is pretty neat,
and compared to their TaskGroup</code> approach for concurrency,
it significantly</em> reduces the amount of code needed for many uses 2</a></sup>.</p>
^2</sup>I didn't realize Swift had a way to concurrently await arrays. It
seems we've independently come up with similar
solutions</a>.
That's probaly a good sign!</p>
</div>
What stands out to me is how easy this makes it to introduce ad-hoc
parallelism into a codebase. The difference between parallel code and
non-parallel code is the addition of a few async</code> keywords sprinkled in front of
existing let</code> statements. The ease of this is really cool, and I think we can
learn something from that!</p>
Postfix spawn</h2>
Now what if we could do this in Rust? Well, not literally</em> the same, but functionally</em>?
What if we could make spawning tasks not much harder than creating futures?
That'd be nice right? - and I'm happy to share I've got something working as
part of the tasky crate</a>. Here's an example of it in use:</p>
use </span>tasky::prelude::*;
</span>
</span>let</span> res = async { "</span>nori is a horse</span>" }.</span>spawn</span>().await;
</span>assert_eq!(res, "</span>nori is a horse</span>");
</span></code></pre>
This will convert an existing future to a task, schedule it on the async-std</code>
runtime, and await the result. But that's not all; when we call spawn</code> it
doesn't actually create a task, it creates an async builder</a>
which can be used to configure the task further before it's started. For
example, we can use it to give our task a name before spawning it:</p>
let</span> res = async { "</span>nori is a loaf of bread</span>" }
</span>    .</span>spawn</span>()
</span>    .</span>name</span>("</span>nori's tuna stash</span>".</span>into</span>())
</span>    .await;
</span>assert_eq!(res, "</span>nori is a loaf of bread</span>");
</span></code></pre>
The way the "async builder" pattern works is by using the nightly IntoFuture</code>
trait. Just like for item in iter</code> loops call IntoIterator</code> on iter</code>. So will
.await</code> call IntoFuture</code> on whatever's being awaited. In this case: the
TaskBuilder</code> type. This enables us to create chaining builders that are
converted into futures when .await</code>ed.</p>
Now what would Swift's earlier example look like if we translated it? For that
we'd need the futures-concurrency</code></a> library too. Let's take a look:</p>
// given: 
</span>//   async fn chop_vegetables() throws -> io::Result<Vec<Vegetables>>
</span>//   async fn marinate_tofu() -> Tofu
</span>//   async fn preheat_oven(temperature: u16) -> Oven
</span>
</span>use </span>tasky::prelude::*;
</span>use </span>futures_concurrency::prelude::*;
</span>use </span>std::time::Duration;
</span>use </span>std::io;
</span>
</span>async </span>fn </span>make_dinner</span>() -> io::Result<Meal> {
</span>    </span>let</span> veggies = </span>chop_vegetables</span>().</span>spawn</span>();
</span>    </span>let</span> meat = </span>marinate_tofu</span>().</span>spawn</span>();
</span>    </span>let</span> oven = </span>preheat_oven</span>(</span>350</span>).</span>spawn</span>();
</span>
</span>    </span>let</span> dish = Dish(&[veggies, meat].</span>try_join</span>().await?);
</span>    oven.</span>cook</span>(dish, Duration::from_secs(</span>60 </span>* </span>60 </span>* </span>3</span>)).await?
</span>}
</span></code></pre>
There are still a few warts; but this gets us real</em> close to having semantics
as smooth as Swift's. The most awkward
pieces of the example are that we don't have Duration::from_hours</code>, and the
array requires a try_join</code> method instead of implementing IntoFuture</code>. But
other than that: it's quite close!</p>
Converting the Swift example is nice to show that we can achieve parity with Swift,
but it doesn't show how how much nicer postfix spawn is over the existing free
functions. For that, let's take a look at what it looks like to spawn a longer
chain of futures on a task:</p>
// free-function, inline variant
</span>let</span> body = task::spawn(
</span>    surf::get("</span>https://foo.bar</span>")
</span>        .</span>header</span>(some_header)
</span>        .</span>header</span>(other_header)
</span>        .</span>body_json</span>(data)
</span>        .recv_json::<MyResponse>()
</span>).await;
</span>
</span>// free-function, variable variant
</span>let</span> req = surf::get("</span>https://foo.bar</span>")
</span>    .</span>header</span>(some_header)
</span>    .</span>header</span>(other_header)
</span>    .</span>body_json</span>(data)
</span>    .recv_json::<MyResponse>();
</span>let</span> body = task::spawn(req).await;
</span>
</span>// postfix method
</span>let</span> body = surf::get("</span>https://foo.bar</span>")
</span>    .</span>header</span>(some_header)
</span>    .</span>header</span>(other_header)
</span>    .</span>body_json</span>(data)
</span>    .recv_json::<MyResponse>()
</span>    .</span>spawn</span>()
</span>    .await;
</span></code></pre>
What we're not showing in this example are the extra imports free-functions will
always require. Having a postfix spawn function would always just work as part
of the Future</code> trait 3</a></sup>.</p>
Having postfix syntax is not only easier to use, it also composes better
with existing async features such as .await</code> and IntoFuture</code>. And removing
minor speedbumps for common operations is a part of API design that should not
be underestimated.</p>
^{3</sup>
The working group is working on providing shared spawn</code>
and Task</code> abstractions from the stdlib. This is very much an API we could provide.</p>
</div>}Design Considerations</h2>
The implementation of tasky</code>'s postfix Future::spawn</code> methods are not yet as
polished as I'd like them to be. I knocked them out in about an hour last
Friday, and I'm out with COVID right now so I don't see myself trying to polish
them anytime soon. Though I do think it might be useful to note what would be
done differently if I had the chance though. So let's take a look at the
definition:</p>
mod </span>sealed {
</span>    </span>/// Sealed trait to determine what type of bulider we got.
</span>    </span>pub trait </span>Kind {}
</span>}
</span>
</span>/// A local builder.
</span>#[</span>derive</span>(Debug)]
</span>pub struct </span>Local;
</span>impl </span>sealed::Kind </span>for </span>Local {}
</span>
</span>/// A nonlocal builder.
</span>#[</span>derive</span>(Debug)]
</span>pub struct </span>NonLocal;
</span>impl </span>sealed::Kind </span>for </span>NonLocal {}
</span>
</span>/// Task builder that configures the settings of a new task.
</span>pub struct </span>Builder<Fut: Future, K: sealed::Kind> { ... }
</span>impl</span><Fut> IntoFuture </span>for </span>Builder<Fut, NonLocal> { ... }
</span>impl</span><Fut> IntoFuture </span>for </span>Builder<Fut, Local> { ... }
</span>
</span>pub trait </span>FutureExt: Future + Sized {
</span>    </span>/// Spawn a task on a thread pool
</span>    </span>fn </span>spawn</span>(</span>self</span>) -> Builder<</span>Self</span>, NonLocal>
</span>    </span>where
</span>        </span>Self</span>: Send,
</span>    {
</span>        Builder {
</span>            kind: PhantomData,
</span>            future: </span>self</span>,
</span>            builder: async_std::task::Builder::new(),
</span>        }
</span>    }
</span>
</span>    </span>/// Spawn a task on the same thread.
</span>    </span>fn </span>spawn_local</span>(</span>self</span>) -> Builder<</span>Self</span>, Local> {
</span>        Builder {
</span>            kind: PhantomData,
</span>            future: </span>self</span>,
</span>            builder: async_std::task::Builder::new(),
</span>        }
</span>    }
</span>}
</span></code></pre>
We provide two methods on Future</code>: spawn</code> and spawn_local</code>. One spawns Send</code>
futures, the other one !Send</code> futures 4</a></sup>. The builder needs to
know what "mode" it's in to spawn the right future using IntoFuture</code>, so to do
that we thread through a generic param which dictates the spawning mode.</p>
The way we currently do this is not ideal. Enum variants aren't types</a> yet, so
the best alternative we have is to emulate this using traits and manually
written types. Though I think, ideally, we'd be able to differentiate based on a
context</a> so that we can be generic over the backend. We don't have a notion yet
of what runtime APIs would look like in the stdlib, so perhaps surfacing this
as two different traits would actually work well.</p>
^4</sup>These are not the only spawning modes possible; I should
write a post about a third mode (Send</code> once, future will not move threads after
that) at some point. For now, check this out:
raftario/async-into-future</a>.</p>
</div>
The other thing this API hasn't covered is the spawn_blocking</code></a> API. This
convers synchronous operations to futures, backed by a thread pool. We could</em>
have a postfix variant of this as well, but I'm not sure whether it's a good
idea. In theory we could have a .spawn</code> method for any type T: !Future</code> using
some sort of trait. But I'm not sure how much value that would provide over a
free function. Personally I'm a bit wary of spawn_blocking</code>, and think if we we
were to expose it from the stdlib, a free function would probably suffice.</p>
Conclusion</h2>
async-std</code> first proposed the model of: "What if tasks are async threads". smol</code> moved away from this by providing "cancel-on-drop" semantics.
In this post we showed some of the benefits gained by moving further away from
that model, and closer to: "What if tasks are parallelizable futures?"</p>
In the future I'd very much like to explore whether we could move
even closer to the "tasks-as-futures" model. Right now the tasky</code>
crate allows people to break structured concurrency guarantees by calling
JoinHandle::detach</code> or mem::forget(handle)</code>:</p>
use </span>tasky::prelude::*;
</span>
</span>async { </span>loop </span>{} } </span>// runs forever on a thread
</span>    .</span>spawn</span>()
</span>    .</span>into_future</span>()
</span>    .</span>detach</span>();
</span></code></pre>
What if we made it so tasks aren't spawned on executors until .await</code>ed, just
like futures? What if we did away with detach</code> entirely? We can't guarantee that
all runtimes behave this way, since people can implement their own. But for the
stdlib, would that make sense? This would still use "cancel on drop semantics"
since .await</code> points mark cancellation sites</a>. But we'd
also gain "tasks aren't started until .await</code>ed", just like futures. Just how
much do we value having structured concurrency? Just what exactly does eager
execution of tasks provide us right now?</p>
Structured concurrency increases safety in concurrent programs,
and I think it's exploring in closer detail what it provides, and what we need
to do to achieve it. But until then: I hope you enjoyed this post! I wrote this because
indexing a bunch of the Swift proposals, learned about
async let</code>, and got excited. And I hope you're excited it now too!</p>

Safe Pin Projections Through View Types
Fri, 04 Mar 2022 00:00:00 +0000
"Pinning" is one of the harder concepts to wrap your head around when writing
async Rust. The idea is that we can mark something as: "This won't change memory
addresses from this point forward". This allows self-referential pointers to
work, which are needed for async borrowing over await points. Though that's what
it's useful for. In practice, how we use it is (generally) through one of three
techniques:</p>

stack pinning</strong> 1</a></sup>: this puts an object on the stack and ensures it
doesn't move.</li>
heap pinning</strong>: using Box::pin</code> to pin a type on the heap to ensure it
doesn't move. This is often used as an alternative to stack pinning.</li>
pin projections</strong>: convert from coarse: "This whole type is pinned" to more
fine-grained: "Actually only these fields on the type need to be pinned".</li>
</ul>
^{1</sup>
Annoyingly that's inaccurate. We can have a future which we later box.
That means we pin on the "stack" only relative to the function, even if the
future itself may later live on the heap. "inline" vs "out of bounds" pinning
may be more accurate. Though if you look at what a "stack" and "heap" are, you
quickly find out those are made-up too. It's all stuff living somewhere, maybe.
I find "stack-pinning" useful way to explain "pinned inside the function body",
so let's just go with that for now, alright?</p>
</div>}“stack pinning” is the less common of the two operations, but we’re on a path to
adding first-class support to it via the core::pin::pin!</code>
macro</a>. For “pin projection” we
don’t have a path towards inclusion yet, and we’re left relying on ecosystem
solutions such as the
pin-project</code></a>
crate2</a></sup> (see footnote for praise).</p>
^{2</sup>
pin-project</code> is an excellent piece of engineering; I want there
to be zero doubt about that. It's what makes the futures ecosystem possible, and
has made it so that people (like myself) can safely</em> author futures by hand
without risking undefined behavior. While this post seeks to explore ways in
which we might move past requiring the need to use pin-project</code>, it's not
because of issues with the crate. It's because I think there might be additional
value in lifting this to the language level in a way that a crate cannot capture
(simply by nature of being a crate).</p>
</div>
The status quo is not bad at all, but it’d be a shame if this is where we
stopped. Pinning is useful for so much more than just futures, and perhaps by
making it more accessible people could experiment with it more. In this post I
want to share the beginnings of an idea I have for how we might be able to add
pin projections as a first-class language feature to Rust.</p>}What are pin projections?</h2>
As we mentioned: pin projections convert from coarse “this whole type is
pinned” to more fine-grained: “Actually only these fields on the type need to be
pinned”.</p>
There’s a thorough explainer about this on the std::pin</code> docs in the
stdlib</a>.
But sometimes it’s better to show rather than tell. So let’s write a quick
example using the pin-project</code> crate. Let’s create a sleep</code> future which uses
some</em> internal timer (work with me here 3</a></sup>), and guarantees that it panics if it’s
polled again after it’s completed:</p>
^{3</sup>
Those familiar enough with runtime implementations might
recognize this API as async-io</code>'s Timer</code> API. This type is Unpin</code>, so if
we were actually using that here we didn't need to perform pin projections for
things to work. But let's pretend that our timer here does</em> need to be pinned
so we can write an example.</p>
</div>}// Our main API which creates a future which wakes up after <dur>
</span>pub fn </span>sleep</span>(</span>dur</span>: Duration) -> Sleep {
</span>    Sleep {
</span>        timer: Timer::after(dur),
</span>        completed: </span>false</span>,
</span>    }
</span>}
</span>
</span>// Our future struct. The `timer` implements `Future`,
</span>// and is the type we want to keep fixed in memory.
</span>// But the `completed` field doesn't need to be pinned,
</span>// so we don't mark it with "pin".
</span>#[</span>pin_project</span>]
</span>pub struct </span>Sleep {
</span>    #[</span>pin</span>]
</span>    </span>timer</span>: Timer,
</span>    </span>completed</span>: </span>bool</span>,
</span>}
</span>
</span>impl </span>Future </span>for </span>Sleep {
</span>    </span>type </span>Output = Instant;
</span>
</span>    </span>fn </span>poll</span>(</span>self</span>: Pin<&</span>mut Self</span>>, </span>cx</span>: &</span>mut </span>Context<'_>) -> Poll<</span>Self::</span>Output> {
</span>        assert!(!</span>self</span>.completed, "</span>future polled after completing</span>");
</span>
</span>        </span>// This is where the pin projection happens. We go from a `Pin<&mut Self>`
</span>        </span>// to a new type which can be thought of as:
</span>        </span>// `{ timer: Pin<&mut Timer, completed: bool }`
</span>        </span>let</span> this = </span>self</span>.</span>project</span>();
</span>        </span>match</span> this.timer.</span>poll</span>(cx) {
</span>            Poll::Ready(instant) => {
</span>                *this.completed = </span>true</span>;
</span>                Poll::Ready(instant.</span>into</span>())
</span>            }
</span>            Poll::Pending => Poll::Pending,
</span>        }
</span>    }
</span>}
</span></code></pre>
If you haven’t seen any manual futures code before, that can be a lot to take
in. It’s a goal of the Rust Async WG to make it so people need to write as
little Pin</code> code as possible to use async Rust. So if this overwhelming to you
/ you’re unsure what’s going on. Please don’t feel pressured you need to learn
this.</p>
But for folks who are curious: the interesting part here is the call to
self.project()</code>. This method is provided by the pin-project</code> crate, and allows
us to convert from Pin<&mut Self></code> to a generated type where only certain
fields are pinned. I like to visualize it for myself like this:</p>
// before projecting
</span>self</span>: Pin<&</span>mut</span> Sleep </span>{</span> timer: Timer, completed: </span>bool</span> }>
</span>
</span>// after projecting
</span>self</span>: &</span>mut</span> Sleep { timer: Pin<&</span>mut</span> Timer>, completed: </span>bool </span>}>
</span></code></pre>
Both provide the guarantee that types which need to be pinned in fact are</em>
pinned. But the “after projecting” example does so on a per-field basis, rather
than locking the whole thing in-place. And if the whole struct is locked in
place, it means we can’t mutate fields on it. Which in turn means we could never
mark completed</code> as true</code>. So we have</em> to project to write stateful futures.</p>
View Types</h2>
In their post “view types for
Rust”</a> Niko
talks about a new language construct called “view types” which can create a
finer-grained interface for borrowing values. The whole post is worth a read,
but to recap the example:</p>
// This is the code we start with
</span>
</span>struct </span>WonkaShipmentManifest {
</span>    </span>bars</span>: Vec<ChocolateBar>,
</span>    </span>golden_tickets</span>: Vec<</span>usize</span>>,
</span>}
</span>
</span>impl </span>WonkaShipmentManifest {
</span>    </span>fn </span>should_insert_ticket</span>(&</span>self</span>, </span>index</span>: </span>usize</span>) -> </span>bool </span>{
</span>        </span>self</span>.golden_tickets.</span>contains</span>(&index)
</span>    }
</span>}
</span>
</span>impl </span>WonkaShipmentManifest {
</span>    </span>fn </span>prepare_shipment</span>(</span>self</span>) -> Vec<WrappedChocolateBar> {
</span>        </span>let mut</span> result = vec![];
</span>        </span>for </span>(bar, i) in </span>self</span>.bars.</span>into_iter</span>().</span>zip</span>(</span>0</span>..) {
</span>            </span>let</span> opt_ticket = </span>if </span>self</span>.</span>should_insert_ticket</span>(i) {
</span>                Some(GoldenTicket::new())
</span>            } </span>else </span>{
</span>                None
</span>            };
</span>            result.</span>push</span>(bar.</span>into_wrapped</span>(opt_ticket));
</span>        }
</span>        result
</span>    }
</span>}
</span></code></pre>
And it yields this error:</p>
error[E0382]: borrow of partially moved value: `self`
</span>   --> src/lib.rs:16:33
</span>    |
</span>15  |         for (bar, i) in self.bars.into_iter().zip(0..) {
</span>    |                                   ----------- `self.bars` partially moved due to this method call
</span>16  |             let opt_ticket = if self.should_insert_ticket(i) {
</span>    |                                 ^^^^ value borrowed here after partial move
</span></code></pre>
However if we change the definition of should_insert_ticket</code> to be more
fine-grained, we can tell the compiler that our borrows do not in fact overlap,
which makes the borrow checker happy.</p>
impl </span>WonkaChocolateFactory {
</span>    </span>fn </span>should_insert_ticket</span>(&{ golden_tickets } </span>self</span>, </span>index</span>: </span>usize</span>) -> </span>bool </span>{
</span>                          </span>// ^ this is the "view type"
</span>        </span>self</span>.golden_tickets.</span>contains</span>(&index)
</span>    }
</span>}
</span></code></pre>
To me the core insight of “view types” is that by encoding more details in type
signatures, the compiler is able to make better decisions about what’s allowed
or not. Who’d have guessed?</p>
View types for pin projection</h2>
So far we’ve looked at “pin projection” and “view types” individually. But what
happens if we put them together? Could we express pin projections using</em> view
types? I think we might! “view types” as proposed by Niko are a way to teach the
compiler about lifetimes in finer detail. I think we could apply the same logic
to use view types to teach the compiler about pinning in finer detail too.</p>
⚠ So before we dive in I want to remind folks again that this is just an idea. I’m
not on the lang team. I am in not in any position to make decisions about this.
I also don't consider myself an expert in pin semantics, so I might have
gotten things wrong. I am not saying that we should prioritize work on this.
I'm sharing thoughts here in the hopes of, well, progressing the language. And
sometimes putting half-baked ideas into writing can help others refine them into
something practical later on.</em> ⚠</p>
Anyway, let’s take our earlier example and try and fit pin projections on it
using view types. Taking it very literally we could imagine the signature of
Future::poll</code> to be possible to express like this:</p>
// base
</span>fn </span>poll</span>(</span>self</span>: Pin<&</span>mut Self</span>>, </span>cx</span>: &</span>mut </span>Context<'_>) -> Poll<</span>Self::</span>Output> { .. }
</span>
</span>// using view types. `timer` is pinned.
</span>fn </span>poll</span>(&</span>mut</span>{ timer, completed } </span>self</span>: Pin<&</span>mut Self</span>>, </span>cx</span>: &</span>mut </span>Context<'_>) -> Poll<</span>Self::</span>Output> { .. }
</span></code></pre>
Okay okay, this is a super literal application of “what if view types would just
do the pin projection”. But bear with me, we’ll go over all details in a second.
For now let’s just assume this works, and slot it into the earlier example:</p>
pub fn </span>sleep</span>(</span>dur</span>: Duration) -> Sleep { .. }
</span>
</span>pub struct </span>Sleep {
</span>    </span>timer</span>: Timer,
</span>    </span>completed</span>: </span>bool</span>,
</span>}
</span>
</span>impl </span>Future </span>for </span>Sleep {
</span>    </span>type </span>Output = Instant;
</span>
</span>    </span>fn </span>poll</span>(&</span>mut</span>{ timer, completed } </span>self</span>: Pin<&</span>mut Self</span>>, </span>cx</span>: &</span>mut </span>Context<'_>) -> Poll<</span>Self::</span>Output> {
</span>         </span>// ^ we're using the view type here
</span>        assert!(!completed, "</span>future polled after completing</span>");
</span>        </span>match</span> timer.</span>poll</span>(cx) {
</span>            Poll::Ready(instant) => {
</span>                *completed = </span>true</span>;
</span>                Poll::Ready(instant.</span>into</span>())
</span>            }
</span>            Poll::Pending => Poll::Pending,
</span>        }
</span>    }
</span>}
</span></code></pre>
I think overall this gets close to the feel we want pin projections to have.
Instead of our base type we narrow it down into its parts. But it’s probably not
everything we want to do yet, and it certainly doesn’t cover all aspects of pin
projections yet.</p>
Ergonomics</h2>
In my opinion the Future::poll</code> signature above is rather noisy; the view type
might have helped simplify the function body and struct declration, but it's
definitely not helped the function signature. And somewhat annoyingly: we don't
have any more information than we started with. We can't actually tell which
fields are pinned!</p>
fn </span>poll</span>(&</span>mut</span>{ timer, completed } </span>self</span>: Pin<&</span>mut Self</span>>, </span>cx</span>: &</span>mut </span>Context<'_>) -> Poll<</span>Self::</span>Output> {
</span></code></pre>
Perhaps the compiler can be clever enough to infer that all !Unpin</code> types in a
view type should remain pinned when viewed. But even if it can, type signatures
are as much for humans as they are for the compiler. And this tells us very
little about what is pinned. Instead I think it’d be clearer if this type of
lowering required contextual pin</code> annotations:</p>
fn </span>poll</span>(&</span>mut</span>{ pin timer, completed } </span>self</span>: Pin<&</span>mut Self</span>>, </span>cx</span>: &</span>mut </span>Context<'_>) -> Poll<</span>Self::</span>Output> {
</span></code></pre>
Now we can tell that the timer</code> is pinned, but completed</code> is not. We finally
have all the detail we need, but the signature is even more overwhelming than it
was. I think it's worth asking at this point whether there's anything we could
remove. If we look at the projected type, we can see there are three parts to it which
tell us what "self" is supposed to look like:</p>
   &</span>mut </span>{ pin timer, completed } </span>self</span>: Pin<&</span>mut Self</span>>
</span>// ^                             ^     ^
</span>// |                             |     This only exists if `self` is this type.
</span>// |                             This is about a `self` param.
</span>// We have a mutable view type with these fields.
</span></code></pre>
There are three parts here, which if we assume we can remove one, we can have
three different permutations:</p>
// view type + self
</span>fn </span>poll</span>(&</span>mut</span>{ pin timer, completed } </span>self</span>, </span>cx</span>: &</span>mut </span>Context<'_>) -> Poll<</span>Self::</span>Output> {
</span>//                                       ^ no right-hand `Pin<&mut Self>`
</span>
</span>// view type + self
</span>fn </span>poll</span>(&</span>mut</span>{ pin timer, completed } pin </span>self</span>, </span>cx</span>: &</span>mut </span>Context<'_>) -> Poll<</span>Self::</span>Output> {
</span>//                                       ^ `Pin<&mut Self>` replaced by `pin self`
</span>
</span>// view type + arbitrary Self-type
</span>fn </span>poll</span>(&</span>mut</span>{ pin timer, completed }: Pin<&</span>mut Self</span>>, </span>cx</span>: &</span>mut </span>Context<'_>) -> Poll<</span>Self::</span>Output> {
</span>//                                  ^ no left-hand `self` type
</span>
</span>// self + arbitrary Self-type
</span>fn </span>poll</span>(</span>self</span>: Pin<&</span>mut Self</span>>, </span>cx</span>: &</span>mut </span>Context<'_>) -> Poll<</span>Self::</span>Output> {
</span>// ... no view type :(
</span></code></pre>
None of these feel particularly great, but they do provide us with some insight.
Pin<&mut Self> can be useful to know _when_ this implementation is available. </code>self` is actually never used since we immediately destructure, but it is useful
to know that this is a method and not a free-function. The view type is what
makes pin projections possible, so we want to keep that somehow.</p>
I'm not sure which attributes make most sense to highlight. The original design
keeps all properties, but feels overwhelming. On reviewing this post, my
co-worker Eric suggested we try and take "view types" proposal, performing
destructuring like we do in patterns, dropping the left-hand variable
declaration entirely:</p>
// self + arbitrary Self-type + view type
</span>fn </span>poll</span>(</span>self</span>: Pin<&</span>mut Self </span>{</span> pin timer, completed }>, </span>cx</span>: &</span>mut </span>Context<'_>) -> Poll<</span>Self::</span>Output> {
</span>
</span>// self + pin Self-type + view type
</span>fn </span>poll</span>(</span>self</span>: pin </span>Self</span> { pin timer, completed }, </span>cx</span>: &</span>mut </span>Context<'_>) -> Poll<</span>Self::</span>Output> {
</span>
</span>// view type + self
</span>fn </span>poll</span>(</span>self</span>: &</span>mut Self</span> { pin timer, completed }, </span>cx</span>: &</span>mut </span>Context<'_>) -> Poll<</span>Self::</span>Output> {
</span>
</span>// view type
</span>fn </span>poll</span>(&</span>mut Self</span> { pin timer, completed }, </span>cx</span>: &</span>mut </span>Context<'_>) -> Poll<</span>Self::</span>Output> {
</span></code></pre>
I think we'll need to keep the self</code> param to not create confusion with Self</code>
methods</a> 4</a></sup>. But
it's hard to say which option out of these would be the best. The second one is
probably the simplest. But the first one conveys more information. Luckily we
don't need to make any decisions now, so we can just settle for having looked at
the space a little.</p>
^{4</sup>
I honestly don't understand why this isn't a concrete method on
Arc</code>. Is this so we don't accidentally hit the Deref</code> implementation? Perhaps
a compiler limitation? I don't really get it, heh.</p>
</div>}Pin projections and drop</h2>
Something we haven’t covered yet is that when pin projecting there are Drop</code>
requirements. Put bluntly, the signature of std::drop::Drop</code> is wrong</em>. If
we’d known about “pinning” before Rust 1.0 we likely would've designed the
Drop</code> trait differently.</p>
The reason why Drop</code> and pinning conflict is because once something has been
pinned it should pinned forever after 5</a></sup>. But Drop</code> takes &mut self</code>, even if a value has previously been pinned</em>, violating this requirement.
And so when we implement Drop</code> for our async types we have to make sure to
always convert &mut self</code> back to self: Pin<&mut Self></code></a>:</p>
^{5</sup>
Unless it's unpinned. But "unpin" is basically: "I don't care
if I'm pinned or not, just do whatever", so let's pretend we're not talking
about it here either.</p>
</div>}impl </span>Drop </span>for </span>Type {
</span>    </span>fn </span>drop</span>(&</span>mut </span>self</span>) {
</span>        </span>// `new_unchecked` is okay because we know this value is never used
</span>        </span>// again after being dropped.
</span>        </span>inner_drop</span>(</span>unsafe </span>{ Pin::new_unchecked(</span>self</span>)});
</span>        </span>fn </span>inner_drop</span>(</span>this</span>: Pin<&</span>mut</span> Type>) {
</span>            </span>// Actual drop code goes here.
</span>        }
</span>    }
</span>}
</span></code></pre>
The pin-project</code> crate covers this case for us. It disallows implementing
Drop</code> manually, and requires we implement
pin_project::PinnedDrop</code></a>
instead, which has the right signature:</p>
use </span>std::pin::Pin;
</span>use </span>pin_project::{pin_project, pinned_drop};
</span>
</span>#[</span>pin_project</span>(PinnedDrop)]
</span>struct </span>PrintOnDrop {
</span>    #[</span>pin</span>]
</span>    </span>field</span>: </span>u8</span>,
</span>}
</span>
</span>#[</span>pinned_drop</span>]
</span>impl </span>PinnedDrop </span>for </span>PrintOnDrop {
</span>    </span>fn </span>drop</span>(</span>self</span>: Pin<&</span>mut Self</span>>) {
</span>        println!("</span>Dropping: </span>{}</span>", </span>self</span>.field);
</span>    }
</span>}
</span>
</span>fn </span>main</span>() {
</span>    </span>let</span> _x = PrintOnDrop { field: </span>50 </span>};
</span>}
</span></code></pre>
If we want to make pinning more accessible, then we cannot just skip over the
interactions with Drop</code>. I'm not sure whether this allows us to trigger
soundness bugs using purely safe Rust - but I suspect the answer to that is
likely irrelevant. It's really easy to get lost in technicalities that we lose
sight of the wider point: Pinning has too many gotchas to use without
pin-project</code>. And it's hard to tease apart to which degree the issues are with
pinning as a concept, or mistakes we've made in the past in the language.</p>
Perhaps "mistakes" sounds too harsh. After all, Pin</code> was introduced after Rust
1.0, so how could we have gotten this right? And while that's true, I think that
perspective is a bit of a dead-end. If the issues are with Pin</code>, then logically
we'll seek to fix Pin</code>. And realizing we're unable to fix Pin</code>, we'll
naturally seek to limit the usage of it. But rather than thinking about pin as
"introducing issues into Rust", what if we instead think of pin as "surfacing
issues we have in Rust".</strong> Pin is a fundamental type in Rust. Transformative
even. It's worth pondering what we could do with it if it was better integrated
into the language.</p>
Maybe we should bite the bullet and actually fix</em> our Drop</code> impl. The main
reason why this hasn't been done is because it's backwards incompatible
6</a></sup>. But likely the most important reason after that is because
it would grossly complicate the ergonomics of implementing Drop</code> for people.
Perhaps that calculation changes if the ergonomics or pinning were less bad.
Say we took one of the ergo examples and applied it to our earlier drop example:</p>
^{6</sup>
As my colleague Eric pointed out, one way we could replace
Drop</code> with a new trait in a backwards-compatible manner is through blanket
impls.
We could define PinnedDrop</code>, and provide a blanket impl of PinnedDrop</code> for T: Drop + Unpin</code>. Then we change the compiler to desugar drops to a pin and a call
to PinnedDrop::drop</code> instead. Though replacing Drop</code> with something else (even
for something as important as soundness) might be tricky to coordinate and
communicate.</p>
</div>}struct </span>PrintOnDrop {
</span>    </span>field</span>: </span>u8</span>,
</span>}
</span>
</span>impl </span>Drop </span>for </span>PrintOnDrop {
</span>    </span>// We've dropped `self: Pin<&mut Self>` from our signature
</span>    </span>// because we immediately destructure it into its parts
</span>    </span>// inside the type signature anyway.
</span>    </span>fn </span>drop</span>(</span>self</span>: pin </span>Self</span> { pin field }) {
</span>        println!("</span>Dropping: </span>{}</span>", field);
</span>    }
</span>}
</span>
</span>fn </span>main</span>() {
</span>    </span>let</span> _x = PrintOnDrop { field: </span>50 </span>};
</span>}
</span></code></pre>
I don't think this looks noticably worse than today's drop trait. And hold onto
your tin-foil caps, I think this could possibly hint at another set of
optimizations we might be able to perform. What if for structs where all types
are Unpin</code> we could create whole-sale projections. u8</code> here doesn't actually
need to be pinned, it's just for show. So what if we actually were able to write
that:</p>
struct </span>PrintOnDrop {
</span>    </span>field</span>: </span>u8</span>,
</span>}
</span>
</span>impl </span>Drop </span>for </span>PrintOnDrop {
</span>    </span>// This is a short-hand for a destructuring of `Pin<&mut Self>`
</span>    </span>// where all fields are `Unpin`, requiring no pinning and thus
</span>    </span>// no destructuring.
</span>    </span>fn </span>drop</span>(&</span>mut </span>self</span>) {
</span>        println!("</span>Dropping: </span>{}</span>", </span>self</span>.field);
</span>    }
</span>}
</span>
</span>fn </span>main</span>() {
</span>    </span>let</span> _x = PrintOnDrop { field: </span>50 </span>};
</span>}
</span></code></pre>
destructuring can be nice. That would allow us to write &mut self</code> as sugar for
self: Pin<&mut Self> where Self: Unpin</code>. Meaning: for all sound</em>
implementations of Drop</code> everything would continue working as they do today.</strong>
The only difference is that the trait declaration of Drop</code> would change. And
we'd catch potential unsound invocations of Drop</code>, requiring them to be sound
at the function signature level. This includes pin-project</code>'s use as well, but
we'd likely be able to coordinate roll-out, deprecation, etc. in the crate.</p>
This is not unlike how &mut self</code> is syntactic sugar for &mut self: &mut Self</code>
already. We could make it so that we can lower Pin<&mut Self></code> to Self</code> if
it's always safe to do so. Anyway. That's probably enough speculation for this
section. To close it off, so far we haven't yet figured out how to introduce
async Drop</code> into the language. One option mentioned so far been to extend
Drop</code> with poll</code> methods</a>. However way
we go about this, we'll have to be looking more closely at Drop</code> and Pin</code> soon
anyway.</p>
#[repr(packed)]</h2>
There are other considerations such as not being able to pin a #[repr(packed)]</code>
type. And honestly: I’m not super sure how we can fix all of this. Maybe we
should raise #[repr(packed)]</code> into the type-system and make Pin<T> where T: !marker::Packed</code>. Or perhaps pin projections should magick their way into
knowing not to do this. I'm not sure. But this is something to figure out if we
want to remove the unsafe {}</code> marker for pin projections.</p>
Unpin</h2>
So far we haven't talked about Unpin</code> at all. I mean, unless you've read the
footnotes. But I assume that's nobody, so here's a recap: Unpin</code> means that a
type simply doesn't care about Pin</code> at all. Moving it around in memory won't
cause it to spontaniously combust, so we can just move it in and out of Pin</code>
wrappers whenever we like. This is true for most types ever created. Only types
which are self-referential (hold pointers to fields contained within itself) are
marked as !Unpin</code>. Which is true for many futures created using async {}</code>,
because that's what borrows across .await</code> points desugar into.</p>
Unpin</code> is an auto-trait: meaning it's implemented for a type if all the fields
of the type implement it. But unlike auto-trait such as Send</code> and Sync</code>, it's
entirely safe</em> to implement. And this causes some issues.</p>
This means that even if our struct contains types which are !Unpin</code>, we can
still manually implement Unpin</code> on it:</p>
use </span>std::marker::PhantomPinned;
</span>use </span>std::marker::Unpin;
</span>
</span>// `Foo` holds an `!Unpin` type, which makes
</span>// `Foo: !Unpin`.
</span>struct </span>Foo {
</span>    </span>_marker</span>: PhantomPinned
</span>}
</span>
</span>// Oh nevermind, we can just mark it as `Unpin` anyway.
</span>impl </span>Unpin </span>for </span>Foo {}
</span></code></pre>
I believe the reasoning is that since "pin projections" require unsafe</code>,
upholding the validity of Unpin</code> should just be done there.
But that's an issue if we're looking for ways to actually make pin projections
safe to perform. Upholding invariants shouldn't be done at the projection site;
it needs to be done on declaration!</p>
The pin-project</code> crate works around this by disallowing safe implementations of
Unpin</code> and instead introducing their own UnsafeUnpin</code>
trait</a>.
This flips the safety requirement back to the Unpin</code> implementation, requiring
invariants are upheld by the implementer of the trait.</p>
use </span>pin_project::{pin_project, UnsafeUnpin};
</span>
</span>#[</span>pin_project</span>(UnsafeUnpin)]
</span>struct </span>Struct<K, V> {
</span>    #[</span>pin</span>]
</span>    </span>field_1</span>: K,
</span>    </span>field_2</span>: V,
</span>}
</span>
</span>// Implementing `Unpin` is unsafe here.
</span>unsafe impl</span><K, V> UnsafeUnpin </span>for </span>Struct<K, V> </span>where</span> K: Unpin + Clone {}
</span>
</span>// Which means calling `struct.project()` later on can be safe.
</span></code></pre>
In order for pin projections to be safe, manual implementations of Unpin</code> need
to be unsafe</code>. This is because someone needs to be responsible for correctly
implementing the invariants; and it either needs to be where we perform the
projection, or where we declare types as Unpin</code>. "both" is annoying. "neither"
is impossible 7</a></sup>.</p>
^{7</sup>
We need to be able to label types as Unpin</code> somehow. Even if
every type in the stdlib implements it, the stdlib itself is not magic. And in
the ecosystem too: core types require being able to be marked with it.</p>
</div>
It seems that if we want to make pin projections a first-class construct in
Rust, we'll have to mark Unpin</code> as unsafe</code>. Annoyingly this would be a change
to the stdlib, and we don't have a process like "editions" where we can flip a
switch on changes like these. At least not yet.</p>
Making this happen wouldn't be easy. And we'd need to assess whether the impact
justifies going down this path. But maybe it does. And maybe we should.</p>}Pin keyword in other places?</h2>
At the start of this post we mentioned that stack-pinning is gaining first-class
support in Rust via the core::pin::pin!</code>
macro</a>. But later on in the post we
also discuss the possibility of improving pin projection semantics by
introducing a pin</code> keyword in the view type. It's worth asking: could a pin</code>
keyword be useful in other places too?</p>
I think, maybe? Even without view types, we could imagine all function
signatures which currently take self: Pin<&mut Self></code> could probably be
simplified to take pin self</code> instead:</p>
// Before
</span>pub trait </span>Future {
</span>    </span>type </span>Output;
</span>    </span>fn </span>poll</span>(</span>self</span>: Pin<&</span>mut Self</span>>, </span>cx</span>: &</span>mut </span>Context<'_>) -> Poll<</span>Self::</span>Output>;
</span>    </span>//            ^ Using an "arbitrary self type"
</span>}
</span>
</span>// After
</span>pub trait </span>Future {
</span>    </span>type </span>Output;
</span>    </span>fn </span>poll</span>(</span>self</span>: pin </span>Self</span>, </span>cx</span>: &</span>mut </span>Context<'_>) -> Poll<</span>Self::</span>Output>;
</span>    </span>//            ^ Replaced by a `pin` keyword.
</span>}
</span></code></pre>
Additionally, for stack pinning we could imagine we could replace the use of
core::pin::pin!</code> with pin-bindings:</p>
// Before
</span>fn </span>block_on</span><T>(</span>fut</span>: impl Future<Output = T>) -> T {
</span>    </span>let mut</span> fut = pin::pin!(fut);
</span>    ...
</span>}
</span>
</span>// Using `let pin` bindings in the function body.
</span>fn </span>block_on</span><T>(</span>fut</span>: impl Future<Output = T>) -> T {
</span>    </span>let</span> pin fut = fut;
</span>    ...
</span>}
</span>
</span>// Using `pin` in the function signature.
</span>fn </span>block_on</span><T>(pin </span>fut</span>: impl Future<Output = T>) -> T {
</span>    ...
</span>}
</span></code></pre>
I don't think this looks particularly out of place. It might be nice even.
But the real question to ask is: is this useful? We have so many things we
could be working on that we have to choose what to prioritize. Right now "pin"
is probably misunderstood and under-used because it doesn't play well with the
language. But if we fixed those issues, would we see people use pin</code> for more
things 8</a></sup>? It's a bit of a chicken/egg problem; though we do
need to have some</em> an answer in order to justify spending time on this.</p>
^{8</sup>
As we mentioned: Pin</code> is what makes borrowing over
.await</code> points work by allowing us to define self-referential structs. But
outside of .await</code> we cannot yet define self-referential structs. If pinning is
required to make that work, maybe there's reason to improve the ergonomics? I'm
genuinely unsure; but I think that's the direction we should be asking questions
in.</p>
</div>}Conclusion</h2>
In this post I’ve shown what pin projections are, what view types are, how the
two could be combined, ergonomics considerations, possible interactions with
Drop</code>, and interactions with Unpin</code>. In general it seems there are 3 questions
to answer?:</p>

Can we use view types to create pin projections? What should that look like?</li>
How can we fix the signature of Drop</code>?</li>
How can we lift #[repr(Packed)]</code> into the type system so we can disallow it to be pinned?</li>
If we want to make pin projections safe, can we mark Unpin</code> as unsafe</code>?</li>
</ul>
Have I missed anything? Are there any other parts to consider? I’m not sure. I
also want to point out that we don’t need a single unifying answer to solve all
of these questions. It may be that each item has a different solution; but once
we solve all of them we can start to realize safe pin projections at the lang
level. As I’ve mentioned a few times now, I’d be keen to hear from others.</p>
Anyway, this is just some stuff I started thinking of after having read Niko’s
view types blog post midway through writing a bunch of futures by hand. I’m not
sure whether delving deeper into this idea is worth it at all right now. As I
mentioned pinning has potential</em>, but we’re trying hard to make users interact
with it as little as possible. Kind of like how we have unsafe</code>, but we don’t
really want people interacting with it much if we can help it, even if the
stakes are different.</p>
From what I’ve heard the Linux kernel folks are making use of pinning for some
of the things they're doing already. I'm not sure what they're doing it for, but
it’d be fascinating</em> to learn why. If people are thinking of improving the
ergonomics of unsafe</code> to make it less hard to use. Perhaps it’d be worth to do
the same for pinning as well?</p>
Thanks to Ryan Levick</a> and Eric Holk</a> for helping review and correct this
post prior to publishing. Niko Matsakis</a> for entertaining my delirious
ramblings about this on Zulip a few days ago. And Boats</a>, with who I'm pretty
sure I've discussed something like this before, even though time is a lie and
at this point I no longer remember how long ago that may have been.</em></p>

More Enum Types
Tue, 15 Feb 2022 00:00:00 +0000
Yesterday I was talking with folks on Zulip about my last blog
post</a>, and
specifically about the potential ergonomic improvements “anonymous enums” could
provide. In my post we only really looked at anonymous enums as a potential
ergonomics improvement for Stream::merge</code>. But in this post I instead want to
dig ever so slightly deeper and show how symmetry</em> is missing in the Rust
language between struct and enum variants.</p>
So I'm writing this post to dig into that some more. We'll be keeping it brief,
focusing mostly on showing something which stood out to me. The intent is less
so for me to be making recommendations about the
direction the language should take, and mostly on walking you through my
observations. There are times when I spend months performing rigorous analysis
of things, and there’s times like these when I just want to share some thoughts.
This post also serves as practice for me to edit less and publish faster.</p>
So with all that out of the way, let’s dig in by looking at structs first.</p>
Structs</h2>
Structs are Rust’s most common data container. They come in three flavors: empty
(no fields), record (key-value), or tuple (indexed fields). All three roughly
work the same, and for the sake of simplicity we’ll only be looking at tuple
structs in this post. So let’s start defining one!</p>
#[</span>derive</span>(Debug, Copy, Clone, PartialEq, Eq, PartialOrd, Ord)]
</span>struct </span>MyStruct(</span>u32</span>, </span>u16</span>);
</span></code></pre>
That’s right, that’s a tuple struct. It has two fields, both of which must be
populated. And it has a bunch of derives, which allow us to perform common,
useful operations on it. Now let’s create one and return it from a function.</p>
#[</span>derive</span>(Debug, Copy, Clone, PartialEq, Eq, PartialOrd, Ord)]
</span>struct </span>MyStruct(</span>u32</span>, </span>u16</span>);
</span>
</span>// named type
</span>fn </span>foo</span>() -> MyStruct {
</span>    MyStruct(</span>100</span>, </span>12</span>)
</span>}
</span></code></pre>
If you’ve ever used a tuple-struct that should look pretty familiar, right? We
created an instance of the type, and returned it from the function. But you
might be noticing that this is actually quite a bit of boilerplate. IDEs can
help us with the derives. But they can’t help us work around defining the type,
the layout, and naming it. What if we didn’t actually care about the name? Say,
if this was just a helper function in our file? For cases where naming the
returned types isn’t useful, we can use tuples:</p>
// anonymous type
</span>fn </span>foo</span>() -> (</span>u32</span>, </span>u16</span>) {
</span>    (</span>100</span>, </span>12</span>)
</span>}
</span></code></pre>
This does almost the exact same thing as the code we had before, but it’s now
much shorter! Gone is the struct declaration. And gone are the derives. This
works because tuples have generic impls on them which say: “if all of my members
implement a trait, then so will I.” You can almost think of it as automatically
deriving traits, while named structs need to declare which traits they want to
derive.</p>
Now finally, sometimes we don’t want to expose which type we return. Often
because we don’t want to provide any stability guarantees about the type we
return. We want to be able to say: “the type we return conforms to this
interface, but that’s all I’m telling you.” For example -> impl Future</code> was
pretty common to author before we had async/.await</code> (in fact async {}</code>
generates an impl Future</code> type, also known as “anonymous future”). But we can
write this in our own code like so:</p>
// type erased type
</span>fn </span>foo</span>() -> impl Debug {
</span>    MyStruct(</span>100</span>, </span>12</span>)
</span>}
</span></code></pre>
All this tells us is: “the type we’re returning here implements Debug</code>, and
that’s it”.</p>
Enums</h3>
Now let’s take a look at enums. Much like structs, enums can carry values as
well (empty, record, tuple) - but instead of being AND (contain multiple
values), they give us XOR (contain an exclusive, single value) 1</a></sup>.
Not sure how much sense this explainer makes, but please pretend it does. In an
enum, only one variant is active at any given time. So let’s define an enum
where we either have a u32</code> or a u16</code>:</p>
^{1</sup>
Yes, "record enums" give us XOR and then AND. But those are
almost like a different kind of shorthand for having named record fields nested
in enums. If we have to go into those we'll never finish this post. So let's
not think about them too much for now.</p>
</div>}#[</span>derive</span>(Debug, Copy, Clone, PartialEq, Eq, PartialOrd, Ord)]
</span>enum </span>MyEnum {
</span>    Left(</span>u32</span>),
</span>    Right(</span>u16</span>),
</span>}
</span></code></pre>
The way we return an enum from a function is by constructing a variant. This is
often times done based on a condition. So let’s pretend we have a function which
is given a condition, and computes which enum to return:</p>
#[</span>derive</span>(Debug, Copy, Clone, PartialEq, Eq, PartialOrd, Ord)]
</span>enum </span>MyEnum {
</span>    Left(</span>u32</span>),
</span>    Right(</span>u16</span>),
</span>}
</span>
</span>// named type
</span>fn </span>foo</span>(</span>cond</span>: </span>bool</span>) -> MyEnum {
</span>    </span>if</span> cond { MyEnum::Left(</span>100</span>) } </span>else </span>{ MyEnum::Right(</span>12</span>) }
</span>}
</span></code></pre>
This will either return the number 100u32</code> or the number 12u16</code>. That makes sense right?</p>
Anonymous enums</h2>
Now what if we wanted to simplify this, say if we were writing a quick helper.
The boilerplate of defining an enum and derives might be distracting, and much
like a tuple we just want to send some data along. We might imagine that we
would want to write this:</p>
// named type
</span>fn </span>foo</span>(</span>cond</span>: </span>bool</span>) -> </span>u32 </span>| </span>u16 </span>{ </span>// made up syntax for u32 OR u16
</span>    </span>if</span> cond { </span>100 </span>} </span>else </span>{ </span>12 </span>}
</span>}
</span></code></pre>
However, Rust doesn’t support this. Unlike “anonymous structs” (tuples), there
is no such thing as an “anonymous enum”. There’s an open issue about
it</a>, but I
don’t know anyone who’s working on it. And I don’t know of any crates which
provide implementations of this either.</p>
Something which makes it difficult to prototype “anonymous enums” is that the
syntax comes in pairs: not only do we need to figure out how to define them in
function return positions, we need to think about how to use them as well. The
version I’ve seen which I like the most used type ascription when matching, but
that’s hard</em> to implement, and has a bunch of other lang implications as well:</p>
match </span>foo</span>(cond) {
</span>    n: </span>u32 </span>=> {},
</span>    n: </span>u16 </span>=> {}
</span>}
</span></code></pre>
Though if someone has ideas how to prototype this, it would be fun to see what people can come up with.</p>
Probably the closest we can get to “anonymous enums” is by using the
either</a> crate from the ecosystem. I
believe once upon a time this crate lived in the stdlib (pre 1.0), and it
provides us with a Result</code>-like type which holds two generic params: one for
the left-hand variant, and one for the right-hand variant. Much like “anonymous
enums”, it allows us to bypass having to provide our own enum by having a
pre-made one.</p>
use </span>either::Either;
</span>
</span>// named type
</span>fn </span>foo</span>(</span>cond</span>: </span>bool</span>) -> Either<</span>u32</span>, </span>u16</span>> {
</span>    </span>if</span> cond { Either::Left(</span>100</span>) } </span>else </span>{ Either::Right(</span>12</span>) }
</span>}
</span></code></pre>
In the wild, types such as
futures::future::Select</code></a>
yield Either</code> variants when wanting to return one of two types. This works
alright when joining two different futures, but when attempting to join more
than two, you’ll end up with nested Join<Join<First, Second>, Third></code>, etc
constructions which are not fun to work with.</p>
In my opinion it would be better if Future::select</code>/ Future::race</code> would
return a single type T</code>, allow race</code> to operate over N futures
concurrently</a>,
and make enum construction an explicit step. That way if Rust ever gets
anonymous enums, the compiler could inference we’re passing different types,
which can be unified using an anonymous enum.</p>
let</span> a = future::ready(</span>1</span>u8</span>);
</span>let</span> b = future::ready("</span>hello</span>");
</span>let</span> c = future::ready(</span>3</span>u16</span>);
</span>
</span>// The compiler could infer the Future's return type T
</span>// should be an anonymous enum, which we can subsequently
</span>// match over
</span>match </span>(a, b, c).</span>race</span>().await {
</span>    n: </span>u8 </span>=> {},
</span>    n: </span>u16 </span>=> {},
</span>    n: &</span>str </span>=> {},
</span>}
</span></code></pre>
In async-std</code> the Future::race</code> method</a> already returns a single type T</code>
instead of an Either</code> type. And for futures-concurrency</code></a> we’re likely going to
do the same 2</a></sup>, but instead of providing a race</code> method from Future</code>, provide a
Race</code> trait directly from container types (though we’re likely going to rename
the traits before we merge them. Current contender: first/First</code> method/trait).</p>
^{2</sup>
The plan is to merge futures-concurrency</code> back into async-std</code> once
we figure out all the parts of it. But that's been taking a minute, and will
likely take a bit longer. But it's important we get this right, sooooo. Hence
the last 3 years of work.</p>
</div>}Type-erased enums</h2>
Now, taking things one level of abstraction further: what if we have N values,
but we only care about the shape of the values returned. We could imagine we
could write something like this:</p>
// type erased type (currently unsupported)
</span>fn </span>foo</span>(</span>cond</span>: </span>bool</span>) -> impl Debug {
</span>    </span>if</span> cond { </span>100</span>u32 </span>} </span>else </span>{ </span>12</span>u16 </span>}
</span>}
</span></code></pre>
Both of these implement Debug</code>, and the syntax is valid this time around. So this should work right? Unfortunately not. We get the following error</a> if we try this:</p>
error[E0308]: `if` and `else` have incompatible types
</span> --> src/lib.rs:4:31
</span>  |
</span>4 |     if cond { 100u32 } else { 12u16 }
</span>  |               ------          ^^^^^ expected `u32`, found `u16`
</span>  |               |
</span>  |               expected because of this
</span>  |
</span>help: you could change the return type to be a boxed trait object
</span>  |
</span>3 | fn foo(cond: bool) -> Box<dyn Debug> {
</span>  |                       ~~~~~~~      +
</span>help: if you change the return type to expect trait objects, box the returned expressions
</span>  |
</span>4 |     if cond { Box::new(100u32) } else { Box::new(12u16) }
</span>  |               +++++++++      +          +++++++++     +
</span></code></pre>
Translating what the compiler is trying to tell us here: “You cannot use impl Debug</code> here, please use Box<dyn Debug></code> instead”. But taking the compiler up on
their suggestion forces us to allocate the value on the heap instead of the
stack, and to go through a pointer vtable to find the right value to dispatch
to. But as we saw earlier: we could also just define an enum to get the same
effect. The suggestion made by the compiler here is sensible, but not optimal</em>.
Though the compiler is not at fault, there’s definitely something missing here.
And I suspect if type-erased enums were supported by the language, the compiler
would never have to suggest how to work around this in the first place, thus
making the issue go away entirely.</p>
Fortunately “type-erased enums” do</em> have a crate in the ecosystem which shows
us what using this would feel like:
auto_enums</a>. By adding
just a few annotations our code suddenly works:</p>
#[</span>auto_enum</span>(Debug)]
</span>fn </span>foo</span>(</span>cond</span>: </span>bool</span>) -> impl Debug {
</span>    </span>if</span> cond { </span>100</span>u32 </span>} </span>else </span>{ </span>12</span>u16 </span>}
</span>}
</span></code></pre>
auto_enums</code> uses a proc macro to generate an enum for each type in
the code. We have to tell it which traits the output type should implement, but
other than that it reads fairly naturally.</p>
Looking at the code generate by the auto_enums</code> crate, I suspect that this
should be fairly straight forward to implement in the compiler if we can get
lang approval for it. Which at first glance doesn’t seem like a big language
feature either. But I don’t know enough about this side of the language to
appropriately judge what gotchas might apply (const, FFI, etc. all need to be
considered). So I don’t know how difficult this might be in practice to see
through to completion. But I’d be interested to hear from any lang/compiler
folks whether my intuition here is right?</p>
Comparing enums and structs</h2>
Now that we’ve taken a look at both structs and enums, we can capture their capabilities in a table:</p>
</th> Structs</th> Enums</th> Enums Fallback</th></tr></thead>

Named</strong></td> struct Foo(.., ..)</code></td> enum Foo { .., .. }</code></td> -</td></tr>
Anonymous</strong></td> (.., ..)</code></td> ❌</td> either</code> crate</td></tr>
Type-Erased</strong></td> impl Trait</code></td> ❌</td> auto_enums</code> crate</td></tr>
</tbody></table>
As you can see, it's much easier to quickly create a struct than it is to create
an enum. Fallbacks for quick enum creation exist in the ecosystem, but
Even if both structs and enums can both hold rich data, the bar to using structs
in Rust is lower than for enums. I would argue, significantly so.</p>
Wrapping up</h2>
We’ve shown the differences between struct and enum definitions. And shown how
the lack of anonymous and type-erased enums is worked around today using crates
from the ecosystem.</p>
My current thinking is that fixing this asymmetry would be beneficial for Rust
eventually. It’s a convenient piece of polish which could make daily driving
Rust nicer. But it’s nothing fundamental that needs to be addressed straight
away. So I don’t think it’s anything which requires being prioritized.</p>
Anyway, I got thinking about this when chatting with folks about this earlier,
and figured I’d put my thinking into words. In part to share the idea, and in
part to learn how to publish things more quickly. If you liked this post, and
would like to see what I make for dinner, follow me on
Twitter</a>.</p>

Appendix A: Examples</h2>
Structs</h3>
struct </span>MyStruct(</span>u32</span>, </span>u16</span>);
</span>
</span>// named type
</span>fn </span>foo</span>() -> MyStruct {
</span>    MyStruct(</span>100</span>, </span>12</span>)
</span>}
</span>
</span>// anonymous type
</span>fn </span>foo</span>() -> (</span>u32</span>, </span>u16</span>) {
</span>    (</span>100</span>, </span>12</span>)
</span>}
</span>
</span>// type erased type
</span>fn </span>foo</span>() -> impl Debug {
</span>    MyStruct(</span>100</span>, </span>12</span>)
</span>}
</span></code></pre>
Enums</h3>
enum </span>MyEnum {
</span>    Left(</span>u32</span>),
</span>    Right(</span>u16</span>),
</span>}
</span>
</span>// named type
</span>fn </span>foo</span>(</span>cond</span>: </span>bool</span>) -> MyEnum {
</span>    </span>if</span> cond { MyEnum::Left(</span>100</span>) } </span>else </span>{ MyEnum::Right(</span>12</span>) }
</span>}
</span>
</span>// anonymous type (currently unsupported)
</span>fn </span>foo</span>(</span>cond</span>: </span>bool</span>) -> </span>u32 </span>| </span>u16 </span>{
</span>    </span>if</span> cond { </span>100</span>u32 </span>} </span>else </span>{ </span>12</span>u16 </span>}
</span>}
</span>
</span>// type erased type (currently unsupported)
</span>fn </span>foo</span>(</span>cond</span>: </span>bool</span>) -> impl Debug {
</span>    </span>if</span> cond { </span>100</span>u32 </span>} </span>else </span>{ </span>12</span>u16 </span>}
</span>}
</span></code></pre>
Appendix B: Tables</h2>
</th> Structs</th> Enums</th> Enums Fallback</th></tr></thead>

Named</strong></td> struct Foo(.., ..)</code></td> enum Foo { .., .. }</code></td> -</td></tr>
Anonymous</strong></td> (.., ..)</code></td> ❌</td> either</code> crate</td></tr>
Type-Erased</strong></td> impl Trait</code></td> ❌</td> auto_enums</code> crate</td></tr>
</tbody></table>

Futures Concurrency III: select!
Wed, 09 Feb 2022 00:00:00 +0000
In our last post on futures concurrency</a> we talked
about the various ways futures in Rust can be awaited concurrently. But we
didn't cover one mode: awaiting multiple futures concurrently and
resolving them as soon as they're ready, without ever discarding any data.
This mode of concurrency is what is conventionally provided by the
futures::select!{}</code> macro</a>.</p>
In this post we're going to take a look at how this mode of concurrency works,
take a closer look at the issues select! {}</code> has, discuss Stream::merge</code> as
an alternative, and finally we'll look at what the ergonomics might look like
in the future. This post is part of the "Futures Concurrency"</em> series. You can find
all library code mentioned in this post as part of the
futures-concurrency</code></a> crate.</p>
process concurrently, yield sequentially</h2>
In our last post we wrote an example where we raced multiple futures, and got
the output from the first future which resolves:</p>
let</span> a = future::ready(</span>1</span>).</span>delay</span>(Duration::from_millis(</span>100</span>));
</span>let</span> b = future::ready(</span>2</span>).</span>delay</span>(Duration::from_millis(</span>200</span>));
</span>let</span> c = future::ready(</span>3</span>).</span>delay</span>(Duration::from_millis(</span>300</span>));
</span>assert_eq!([a, b, c].</span>race</span>().await, </span>1</span>);
</span></code></pre>
Using race</code> gets the future a</code> as soon as it resolves, but discards futures b</code>
and c</code>, making their content unavailable. If we don't want to discard any futures
we can use join</code>, but this requires waiting until all</em> futures have been resolved
before we can operate on their output. What if we wanted something that sits
between the two: How can we get the result of all futures, but process them
one-by-one as soon as they've resolved?</p>
If we pull up the table we introduced last post, we can now add a third column.
Instead of waiting for all</em> outputs or just the first</em> output,
we're now handling all outputs one-by-one, as soon as they're ready:</p>
</th> Wait for all outputs</strong></th> Wait for first output</strong></th> Handle output one-by-one</strong></th></tr></thead>

Continue on error</strong></td> Future::join</code></td> Future::try_race</code></td> ???</code></td></tr>
Return early on error</strong></td> Future::try_join</code></td> Future::race</code></td> ???</code></td></tr>
</tbody></table>
This is the table we've been working with so far, and it makes sense to continue
filling it out. But maybe if we disregard "fallibility" for a moment, there's
another table we can create? Like race</code> we start handling items as soon as the
first item is resolved, but instead of discarding the remaining items, like
join</code> we are interested in all</em> items:</p>
</th> Handle all items</strong></th> Discard some items</strong></th></tr></thead>

Response starts on first item</strong></td> ???</td> Future::race</code></td></tr>
Response starts on last item</strong></td> Future::join</code></td> ???</td></tr>
</tbody></table>
For the majority of this post we'll be focusing on the top-left of this new
table: "response starts on first item" and "handle all items". We'll cover the
bottom right mode of concurrency towards the end of this post.</p>
Let's start by taking a look at how the futures</code> crate exposes this mode of
concurrency first.</p>
the futures::select! macro</h2>
The futures</code> crate has a solution for this mode of concurrency through the
select!{}</code>
macro</a>. The
tokio</code> runtime also has a variant of select! {}</code></a>, but it behaves
slightly differently. Both the futures</code> and tokio</code> variants of select</code> behave
largely the same, so the broader points we're making apply to both. However
where relevant, we will look closer at how the two differ.</p>
The select! {}</code> macro introduces a custom DSL</a> which resembles match</code> blocks,
but with a few twists. Unlike a match</code> block which takes a single input,
select! {}</code> takes multiple futures, creates named bindings to them when they
resolve, and executes a block when it executes. It also has a complete</code> case
(else</code> in the tokio</code> variant) which runs when no more data is available from
any of the input futures.</p>
Here's a basic example of how select!{}</code> is used, adapted from the async
book</a>.
This takes multiple asynchronous streams of numbers and sums them (playground</em></a>):</p>
use </span>futures::stream::{</span>self</span>, StreamExt};
</span>use </span>futures::select;
</span>
</span>// Create two streams of numbers. Both streams require being `fuse`d.
</span>let mut</span> a = stream::iter(vec![</span>1</span>, </span>2</span>, </span>3</span>]).</span>fuse</span>();
</span>let mut</span> b = stream::iter(vec![</span>4</span>, </span>5</span>, </span>6</span>]).</span>fuse</span>();
</span>
</span>// Initialize the output counter.
</span>let mut</span> total = </span>0</span>;
</span>
</span>// Process each item in the stream;
</span>// break once there are no more items left to sum.
</span>loop </span>{
</span>    </span>let</span> item = select! {
</span>        item = a.</span>next</span>() => item,  </span>// 1. Get the next future in the stream,
</span>        item = b.</span>next</span>() => item,  </span>//    assign its `Output` to `item` when it resolves.
</span>        complete => </span>break</span>,        </span>// 2. The `complete` case runs when both streams are exhausted. 
</span>    };
</span>    </span>if let </span>Some(num) = item {
</span>        </span>// Increment the counter with the value from the stream.
</span>        total += num;             
</span>    }
</span>}
</span>
</span>// Validate the result.
</span>assert_eq!(total, </span>21</span>);
</span></code></pre>
This example is simplified. It can roughly be broken into three steps:</p>

Create the streams and fuse</code></a> them. fuse</code> makes it so once a Stream</code>
returns None</code>, it guarantees to keep returning None</code> instead of potentially panicking 1</a></sup>.</li>
Create the main processing loop using select!</code>, summing the numbers together.</li>
Validate the result is correct.</li>
</ol>
^{1</sup>
Unfused streams are not guaranteed to panic if they are accessed again
after they've been exhausted. But they definitely can</em> panic, and it's not
unlikely that when we have generators they will, just like Future</code>, panic if
they're called again after they've been exhausted.</p>
</div>}Issues with futures::select!</h2>
Now that we've talked about the select! {}</code> macro and the mode of concurrency
it enables, it's time we start looking at the issues with it. Because oh boy are
there many. My personal opinion has always been that select! {}</code> seems hard to
use, and feels off. But in writing this post I've had to substantiate this
feeling with actual evidence, and I don't like what I've learned. Not. At. All.</p>
In this section we'll cover different aspects of how select! {}</code> falls short.
But if you want my brief, summarized opinion: I view select! {}</code> as easy to
misuse, hard to debug, difficult to learn, when used correctly</em> makes for
incredibly hard-to-read code, and seems unlikely to being included in the stdlib
or language.</strong> I think this qualifies as a strong statement</em>. Now let me back
it up by sharing my reasoning for each part.</p>
modifying "hello world" can lead to data loss</h3>
select! {}</code> is not limited to implementing a single form of concurrency: it is
flexible enough that it can be used to implement different concurrency
primitives. This makes it hard for both the compiler and humans to validate
whether it's used correctly. Which is not great when performing concurrent
operations in a potentially multi-threaded system. This can be exceptionally
hard to debug.</p>
Let's take our earlier example, and instead of taking numbers from a stream and
incrementing a counter, let's do something we might encounter in the real-world.
For example, in one select arm we may want to process items from a stream, and in
the other we may want to read data from some handle and send it into a channel:</p>
// before
</span>loop </span>{
</span>    </span>let</span> item = select! {
</span>        item = a.</span>next</span>() => item,
</span>        item = b.</span>next</span>() => item,
</span>        complete => </span>break</span>,
</span>    };
</span>    ...
</span>}
</span></code></pre>
// after
</span>loop </span>{
</span>    futures::select! {
</span>        _ => </span>read_send</span>(&</span>mut</span> handle, &</span>mut</span> channel).</span>fuse</span>() => {} </span>// read data from a handle and send it to a channel
</span>        item => stream.</span>next</span>() => { ... }                       </span>// process data from a stream
</span>        complete => </span>break</span>,                                     </span>// all work completed, stop the loop
</span>    }
</span>}
</span></code></pre>
If you squint you can see that the shape of both loops remained roughly the
same. The most significant change is that instead of calling stream.next()</code>
in both arms, we have switched out one arm to use read_send</code> instead. We can no
longer rely on the futures being fused, so we need to manually do that on the
future now instead. But what you probably didn't expect is that this code is
fundamentally broken, and your program will</em> lose data.</p>
The second example is the exact pattern Tomaka covers in their 2021 post on async
Rust</a>. They based this post on their experience writing production async
Rust code for several years, and described this issue as follows (emphasis is
Tomaka's):</p>

It is always possible to solve this problem in some way, but what I would like
to highlight is an even bigger problem: these kind of cancellation issues are
hard to spot and debug</strong>. In the problematic example, all you will observe is that
in some rare occasions some parts of the file seem to be skipped. There will not
be any panic or any clear indication of where the problem could come from. This
is the worst kind of bugs you can encounter.</strong></p>
</blockquote>
What's happening here is that we intended to express: "process concurrently,
yield sequentially" semantics. But what we ended up writing was "race"
semantics. The reason for this is that read_send</code> and stream.next()</code> are
racing, and if stream.next()</code> completes first, the read_send</code> future is
discarded. This can lead to data from handle</code> having being read, but not yet
sent into the channel</code>. Which means the next time we read data from handle</code>,
the cursor will have advanced, and data is now lost.</p>
You may be wondering why stream.next()</code> does</em> work, but read_send</code> does not.
The short explanation is that the state of the stream.next</code> future is entirely
contained within stream</code>. Which means that if we re-create the future on each
iteration, we can continue right where we left off. read_send</code> on the contrary
has state local to the future. Which means when we recreate the future, we lose
that state, regardless of how far we got.</p>
In their post Tomaka appears to primarily attribute the issue they're
experiencing to all futures being cancellable. And that makes sense: if future
can't be cancelled, it can't lead to data loss. And if select!</code> only ever
operates un cancellable futures, then we've successfully guarded against this
issue! But in my opinion the issue is less with all futures being cancellable
(I think this is
good</a>),
but instead that depending on how you configure select!</code> you may
unintentionally be presented with "race" semantics</a>!</p>
In general I'd recommend folks don't attempt to use select!</code> for this mode of
concurrency and instead spawn (local) tasks instead. This is
largely how over the years I've avoided to ever have to write select!</code> statements using
futures. But because we're talking about the issues with select!</code> we
should at least take a look at what a correct solution using select!</code> might
look like:</p>
// Create the future we're polling once, outside the loop so it
</span>// doesn't drop between iterations.
</span>let mut</span> read_send_fut = </span>read_send</span>(&</span>mut</span> handle, &</span>mut</span> channel);
</span>loop </span>{
</span>    futures::select! {
</span>        _ => read_send_fut => {
</span>            </span>// Whenever the last future resolves, we need to re-create it for the next loop.
</span>            read_send_fut = </span>read_send</span>(&</span>mut</span> handle, &</span>mut</span> channel);
</span>        }
</span>        item => stream.</span>next</span>() => { ... }
</span>        complete => </span>break</span>,
</span>    }
</span>}
</span></code></pre>
Let me be honest: I don't feel good about this solution. We're instantiating the
exact same future in two separate places. And because this is an example, we're
lucky in this case that we only do this for the one future. In real-world
examples we might want to repeat this more often. We could probably abstract the
future instantiation using a function. But I'm hesitant to introduce further
abstractions.</p>
To me this code feels brittle, riddled with assumptions which are not directly
obvious from the code, and it's becoming hard to see what it is that we're
actually</em> trying to do here. And that's an issue. select!</code> is so flexible, it
stops being able to accurately capture intent</em>. This makes it impossible for
the compiler to validate we're using it right. And hard for people reading the
code to understand what we're trying to do. Because it's possible to mis-use
select!</code> it means that any diligent code-reviewer will manually need to
validate select!</code> blocks are right, every time they're modified. Because the
consequences of failing to do so can have very real consequences.</strong></p>
An alternative to select! {}</code> should only expose a single concurrency mode from
its API, making it impossible to accidentally opt-in to a different mode. One of
Rust's core strengths is the ability for the compiler to validate our usage of
APIs is correct. And for something as fundamental as a concurrency mode we should
absolutely be making use of this.</p>
halt points are hidden in a macro</h3>
In my post on async cancellation</a>, I
showed the following example displaying every point where code can be halted.
You can think of "halting" as "this is where the function will potentially stop
executing, so we need to think about how to clean up local state when it does":</p>
// Regardless of where in the function we stop execution, destructors will be
</span>// run and resources will be cleaned up.
</span>async </span>fn </span>do_something</span>(</span>path</span>: PathBuf) -> io::Result<Output> {
</span>                                        </span>// 1. the future is not guaranteed to progress after instantiation
</span>    </span>let</span> file = fs::open(&path).await?;  </span>// 2. `.await` and 3. `?` can cause the function to halt
</span>    </span>let</span> res = </span>parse</span>(file).await;        </span>// 4. `.await` can cause the function to halt
</span>    res                                 </span>// 5. execution has finished, return a value
</span>}
</span></code></pre>
Something which makes select! {}</code> difficult to reason about with regards to
halting (and cancellation) is that it obfuscates potential halt points by not requiring that they
are marked with .await</code>:</p>
loop </span>{
</span>    </span>let</span> item = select! {
</span>        item = a.</span>next</span>() => item, </span>// 1. .await is hidden here, which can cause a halt
</span>        item = b.</span>next</span>() => item, </span>// 2. .await is hidden here, which can cause a halt
</span>        complete => </span>break</span>,
</span>    };
</span>    ...
</span>}
</span></code></pre>
Generally the rule is that halt points contained in function bodies are
annotated using ?</code> or .await</code>. But select!</code> hides .await</code> statements
internally, meaning they can no longer be seen in in the function body. Someone
unfamiliar with select!</code>'s semantics may not expect it to cause the function to
halt, which can lead to bugs. An alternative to select! {}</code> should use .await</code>
to annotate it can halt.</p>
fuse</code> requirements</h3>
When a Future</code> is created through using the async</code> keyword, if it's polled after
it's been completed, it will panic. Similarly, once we have generators in the
language, creating a Stream</code> through an async gen fn</code> we would logically
expect that to panic too if we call it after it's been completed 2</a></sup>.</p>
^{2</sup>
The compiler guards against "poll-after-completion" errors in
most cases because .await</code> takes ownership of futures. This means if you call .await</code>
twice on the same future, it'll fail to compile with a "use of moved value"
error. The "panic if this future is polled after it's completed" error is kind
of like the runtime version of the move semantics guard, for when a future is
manually polled to completion. While not required by the Future</code> trait, I think
it's good practice for most futures to implement similar "poll-after-completion"
behavior as futures created through the async</code> keyword.</p>
</div>
"fuse semantics" mean slightly different things for futures, iterators, and streams:</p>}
Future</code></strong>: after a future has returned Poll::Ready</code> once, keep returning
Poll::Pending</code> instead of panicking(ref</a>).</li>
Iterator</code></strong>: after an iterator has returned None</code> once, keep returning None</code>
instead of panicking (ref</a>).</li>
Stream</code></strong>: after a stream has returned Poll::Ready(None)</code> once, keep
returning Poll::Ready(None)</code> instead of
panicking (ref</a>).</li>
</ul>
Futures passed to select!</code> must implement fuse semantics so that we can track
the completion status of each individual future. If we poll a future and it This cannot
be tracked inside the select! {}</code> macro because select! {}</code> does not take
ownership of individual futures. Once a select! {}</code> block has yielded, the
futures within it can be accessed again, which is often used to pass them back
into another select! {}</code> macro on a subsequent loop.</p>
"Does my future implement fuse semantics" is checked during compilation in the
futures::select!</code>
macro</a>,
but left as an exercise to the user when using
tokio::select!</code></a>.
This makes the futures</code> variant more resilient than the tokio</code> variant, but in
both cases it's annoying that we have to care about this at all because Future</code>
and async/.await</code> were not designed to default to implement fuse semantics.</p>
This mismatch causes select! {}</code> requires users to need to educate themselves
on what fusing futures is, how to do it, and how not to be caught by runtime
errors. select! {}</code> as an abstraction does not fit well with the rest of async
Rust, and as a result users end up needing to learn how to work around this
mismatch. Which does not make for a great experience, and would be great if we
could evade front-loading when teaching folks about async Rust.</p>
trait signatures</h3>
select! {}</code> can not only operate on futures, it can operate on asynchronous
streams too. There is no real counterpart to Future</code> in the stdlib (the syn
version of "asynchronous value" is just "a value"), but the counterpart to
AsyncIterator</code> (née: Stream</code>) is Iterator</code>. An unfortunate consequence of
select! {}</code> is that it has lead to a mismatch between the signatures of
futures::FusedStream</code></a>
and FusedIterator</code></a>.
FusedStream</code> has a mandatory method: is_terminated</code> which needs to be
implemented. FusedIterator</code> does not have this method.</p>
The is_terminated</code> method on Future</code> allows a third state to be represented in
the Future</code> poll model: "this future has completed". For Stream</code> we can
represent this state by yielding Poll::Ready(None)</code>. But in Future</code> we only
have the choice between Poll::Ready(T)</code> and Poll::Pending</code>, neither of which
means "I'm done". As remarked in the original issue discussing FusedFuture</code></a>,
the need for having the is_terminated</code> method would go away if select! {}</code>
would have an internal loop</code> statement. If the loop lives internally, state
tracking can be moved interally as well, and we no longer have the same issues.
But unfortunately select_loop!</code> was deemed as being too
inflexible</a>,
and instead the choice was made to go with FusedFuture</code> / FusedStream</code>
instead 3</a></sup>.</p>
^{3</sup>
select_loop!</code> is so close</em> to what we're proposing as
the solution in this post. I only found out about it towards the end of editing
this post, but it feels like this was just a few iterations away from what we're
proposing we do here instead.</p>
</div>}The reason why discussing trait mismatches is important is because a guiding principle for of the async
foundations WG is parity between sync and async
Rust</a>:</p>

Async Rust should be a small delta atop Sync Rust. People who are familiar
with sync Rust should be able to leverage what they know to make adopting Async
Rust straightforward. Porting a sync code base to async should be relatively
smooth: just add async/await, adopt the async variants of the various libraries,
and you're done.</p>
</blockquote>
Introducing FusedFuture</code>, FusedStream</code> and select! {}</code> into the stdlib would
mean we'd need to take one of three decisions:</p>

We accept that FusedStream</code> and FusedIterator</code> have a different signature,
and all the consequences that carries regarding parity with std
rust</a>.</li>
We re-work select! {}</code> to not rely on FusedFuture::is_terminated</code>.
I believe this has been tried though, and it didn't work out.</li>
We re-work select! {}</code> to not require fuse semantics. This would require
select! {}</code> take ownership of futures, which would lead to data loss.</li>
</ol>
select! {}</code>'s relationship to the Fused</code> traits is not good, and all options
how to resolve it carry distinct downsides. The best solution would be to look
for alternatives to select! {}</code> which implement the same concurrency mode, but
do not rely on fuse semantics on the types they operate on.</p>
debugging errors is hard</h3>
If you misuse select! {}</code>, how do you find out? Implementing race</code> semantics
where you didn't mean will lead to data occasionally</em> being discarded.
Depending on the data you're reading this can lead to hard-to-detect bugs.
There's no guarantees errors will be thrown, it's not obvious from the code
that we're racing, and we have few (if any) tools or lints to detect races like
these. The best chance at catching issues with this before they occur is to have a
thorough test suite which can pinpoint this error before it's shipped. But in my
experience this is not a given.</p>
Debugging work is often 90% understanding the bug, and 10% fixing the bug. And
in order to catch bugs with select! {}</code> you need to understand how the macro
works, how async cancellation works in Rust, find out exactly where it's used,
and finally: employ a fix. This is the type of work that even experts in async
Rust might struggle with (hi).</p>
It doesn't mean that we can't do anything about this though. We could apply
runtime probing to find out where futures are being cancelled in a loop. Or
static lints which catch potential mis-configurations of select! {}</code>. But
that's speculation, and will never be as good a fix as having APIs where bad
states are unrepresentable.</p>
select!</code> and the stdlib</h3>
Objectively select! {}</code> has many other issues, the first being its relation to the
stdlib. As we're looking to add more async functionality to the
stdlib</a>,
foundational components for async concurrency will be part of this. So it's
worth asking how we could add select! {}</code>'s functionality to the stdlib.</p>
Introducing select! {}</code> as a macro would be the first option. But if we
introduced it today, it would immediately become one of the most complicated
macros in the stdlib. select! {}</code> really is shaped like a control-flow
primitive, and Rust tends to prefer keywords over macros for control flow
4</a></sup>. Rust cares deeply about diagnostics, and for something as
fundamental as control flow we would want to provide comprehensive errors,
explainers, and suggestions.</p>
^{4</sup>
The try!</code> macro became ?</code>. The await!</code> macro became .await</code>.
Control flow constructs in Rust tend to be introduced as keywords
instead, but the bar for those is incredibly high.</p>
</div>}Adding select</code> as a keyword to Rust would possibly be harder; the barrier for
keywords in Rust is incredibly high. In order for a new keyword to be
introduced, an irrefutable case must be presented. For .await</code> we
proved that it enabled things that could otherwise only be done using unsafe</code>.
const</code> in generic position were added because it enabled computation within the
type system. select</code> in comparison merely adds a fifth mode of concurrency to
async Rust 5</a></sup>. This is hardly comparable in impact. Especially if we can
prove that we can provide the same concurrency primitives entirely using library
functions.</p>
^5</sup>crossbeam_channel::select</code></a>
exists for non-async Rust and is similar in shape. But it operates
over channels only, and cannot yet be mixed with async Rust. Any
proposal for a select</code> keyword would need to carefully consider
how select</code> would interact with both async and non-async Rust.</p>
</div>
concurrent stream processing with Stream::merge</code></h2>
So far we've taken a look at the mode of concurrency we're trying to implement,
and the issues the select! {}</code> macro has. This is the part where we move past
that and show a surprisingly simple alternative solution to this type of
concurrency: Stream::merge</code>. We'll be stepping away from futures for a second
to talk about streams, but we'll get back how this relates to futures as well in
a bit.</p>
Fundamentally what we want to be doing is awaiting multiple streams concurrently;
processing items from either stream as soon as they're ready. In async-std</code>
we introduced Stream::merge</code> about two years ago (!) for this exact purpose.
It now also exists in futures-concurrency</code>, the companion library to this
series. But for this example we'll stick to async-std</code>.</p>
The way Stream::merge</code> works is that it takes two streams, and merges</em> them as
if they were a single stream. Data comes out as soon as it's ready, and can be
handled just like any other stream:</p>
use </span>async_std::prelude::*;
</span>use </span>async_std::stream;
</span>
</span>// Declare our streams.
</span>let</span> a = stream::iter(vec![</span>1</span>, </span>2</span>, </span>3</span>]);
</span>let</span> b = stream::iter(vec![</span>4</span>, </span>5</span>, </span>6</span>]);
</span>
</span>// Initialize the output counter.
</span>let mut</span> total = </span>0</span>;
</span>
</span>// Combine both streams, and add each value to the total.
</span>a.</span>merge</span>(b).</span>for_each</span>(|</span>num</span>| total += num).await;
</span>
</span>// Validate the result.
</span>assert_eq!(total, </span>21</span>);
</span></code></pre>
This code is functionally equivalent to the select! {}</code> loop we introduced
at the start of this post. Except it's significantly</em> less code, does not require
manually fuse</code>ing streams (Merge</code> tracks the state), and importantly: does
not introduce any new syntax [^gripe].</p>
This is not even the shortest version this can be, using the
stream::Sum</code></a> trait
we can shorten this specific example to:</p>
use </span>async_std::prelude::*;
</span>use </span>async_std::stream;
</span>
</span>let</span> a = stream::from_iter(vec![</span>1</span>, </span>2</span>, </span>3</span>]);
</span>let</span> b = stream::from_iter(vec![</span>4</span>, </span>5</span>, </span>6</span>]);
</span>assert_eq!(a.</span>merge</span>(b).</span>sum</span>().await, </span>21</span>);
</span></code></pre>
Conceptually this seems so simple, it almost feels too good to be true. But it
isn't: there are no hidden Unpin</code> bounds, no secret FusedStream</code> requirements,
or unexpected gotchas. We got to this solution by closely analyzing the mode of
concurrency select! {}</code> exposes, and finding prior art for it in
rxjs</a>.</p>
Concurrently processing a mix of futures and streams</h2>
Perhaps you're wondering whether Stream::merge</code> can only replace select! {}</code>
loops in simple cases. What about differently-typed streams with different
signatures and individual futures mixed in. That's exactly what select! {}</code> was
designed for. For the sake of brevity I'll skip the select! {}</code> variant and
only show how well Stream::merge</code> is capable of handling this6</a></sup>:</p>
^{6</sup>
This reminds me we made mistakes in async-std</code>. stream::once</code> really
should be able to take a Future</code> rather than a regular value. This example
pretends we didn't make that mistake.</p>
</div>}use </span>async_std::prelude::*;
</span>use </span>async_std::stream;
</span>
</span>// Create a shared output enum, used by all stream.
</span>enum </span>Message {
</span>    Num(</span>u8</span>),
</span>    Text(&</span>'static str</span>),
</span>}
</span>
</span>// Create a variety of streams from futures and iterators, and map them to the
</span>// same output type
</span>let</span> a = stream::from_iter([</span>1</span>, </span>2</span>, </span>3</span>]).</span>map</span>(Message::Num);
</span>let</span> b = stream::from_iter(["</span>chashu</span>", "</span>nori</span>"]).</span>map</span>(Message::Text);
</span>let</span> c = stream::once(async { "</span>hello</span>" }).</span>map</span>(Message::Text);
</span>
</span>// Merge all streams and handle each output one-by-one.
</span>let mut</span> s = a.</span>merge</span>(b).</span>merge</span>(c);
</span>while let </span>Some(msg) = s.</span>next</span>().await {
</span>    </span>match</span> msg {
</span>        Num(n) => println!("</span>received a number: </span>{}</span>", n),
</span>        Text(s) => println!("</span>received a string: </span>{}</span>", s),
</span>    }
</span>}
</span></code></pre>
We can even add short-circuiting by using a generalized abort mechanism like
stop-token</code></a>, or adding
another enum variant that maps to break</code> whenever called.</p>
This shows how Stream::merge</code> implements the "process items as soon
as they're available" mode of concurrency, the same as select! {}</code>. And
importantly, it does so using regular Rust types and techniques.</p>
The least comfortable part of what we've written is type unification using enum Message {}</code> and map(Message::Type)</code>. But making sure types align is a common
Rust issue, and using an enum for that is the common solution. The
semantics here are a lot less special</em> than those introduced by select! {}</code>,
which should make this significantly easier to get the hang of. Especially if we
document it right.</p>
If we're thinking about a future where the stdlib provides a complete</em> solution
for futures concurrency out of the box, then the options for select! {}</code> look
rather unappealing. We could choose to ship the macro as-is, and forever deal
with manual fuse</code> calls, diverging traits
[1</a>,
2</a>], and daunting errors.</p>
Or we could promote select</code> into a keyword. This could do away with the
requirement to manually invoke fuse</code> for us. But would come with the downside
that we now have a syntax which is unlike anything else. The arms are different
from those found in match</code>, and it would likely be uniquely scoped to async
contexts. Introducing new keywords is an incredibly</em> high barrier to clear, and
have my doubts whether select</code> could make it.</p>
Concurrently processing futures in a loop</h2>
Earlier in this post we looked at Tomaka's post which featured data-loss issues with
select! {}</code></a>.
Let's bring up the example verbatim from the post:</p>
// open our IO types
</span>let mut</span> file = ...;
</span>let mut</span> channel = ...;
</span>
</span>// This loop re-creates the `read_send` future on each iteration. That means if
</span>// `socket.read_packet` completes, `read_send` may have read data from the file,
</span>// but not finished writing it to a channel, which can lead to data loss
</span>loop </span>{
</span>    futures::select! {
</span>        _ => </span>read_send</span>(&</span>mut</span> file, &</span>mut</span> channel) => {}, </span>// uh oh, data loss can happen here
</span>        some_data => socket.</span>read_packet</span>() => {
</span>            </span>// ...
</span>        }
</span>    }
</span>}
</span></code></pre>
The issue was that we accidentally opted into "race" semantics
when we didn't mean to, which could lead to data loss. In their post, Tomaka
mentions there are different ways of solving this, including my personal choice:
spawn each individual operation on a separate task and await
them</a>
But for the sake of example, let's take a look at how we can implement the logic above using Stream::merge</code>:</p>
// open our IO types
</span>let mut</span> file = ...;
</span>let mut</span> channel = ...;
</span>
</span>// Create our shared message type
</span>enum </span>Message {
</span>    None,
</span>    Data<Vec<</span>u8</span>>>,
</span>}
</span>
</span>// Create two streams by creating and awaiting futures in a loop. We unify the output
</span>// of both futures through the `Message` enum.
</span>let</span> a = stream::repeat_with(|| async { </span>read_send</span>(&</span>mut</span> file, &</span>mut</span> channel).await }).</span>map</span>(Message::None);
</span>let</span> b = stream::repeat_with(|| async { socket.read_packet.await }).</span>map</span>(Message::Data);
</span>
</span>// Merge both the streams and wait for their output one-by-one.
</span>let mut</span> s = a.</span>merge</span>(b);
</span>while let </span>Some(msg) = s.</span>next</span>().await {
</span>    </span>match</span> msg {
</span>        Message::None => </span>continue</span>,
</span>        Message::Data(s) =>  ...,
</span>    }
</span>}
</span></code></pre>
This creates an endlessly looping stream for sending data from the file into the
channel, and another endlessly looping stream to read data from the socket. It's
definitely more verbose than the original select! {}</code> example, but we also have 100%
less data loss. Instead of dropping and re-creating read_send</code> each time
read_packet</code> makes progress, both just keep moving forward the way you would
expect them to.</p>
Would I write this for a project I'm working on? Probably not; I'd likely use
async-task-group</code></a>
instead. But I think this is helpful to show a practical example of a
well-publicized issue with select! {}</code> and how Stream::merge</code> would help guard
against that too.</p>
Fairness</h2>
There is one issue with merge</code> we haven't covered yet: fairness. Let's look at
the example which merges three streams into one:</p>
let</span> a = stream::repeat(</span>1</span>).</span>take</span>(</span>3</span>);
</span>let</span> b = stream::repeat(</span>2</span>).</span>take</span>(</span>3</span>);
</span>let</span> c = stream::repeat(</span>3</span>).</span>take</span>(</span>3</span>);
</span>let</span> d = stream::repeat(</span>4</span>).</span>take</span>(</span>3</span>);
</span>
</span>let</span> s = a.</span>merge</span>(b).</span>merge</span>(c).</span>merge</span>(d);
</span></code></pre>
Streams a</code>, b</code>, and c</code> will yield a number exactly three times. Because they
always have an item available futures will always immediately yield the data
that they have, instead of breifly waiting. Which means that if we implement our
merge</code> function without accounting for this, we would end up with:</p>
assert_eq!(s.</span>collect</span>(), vec![</span>1</span>, </span>1</span>, </span>1</span>, </span>2</span>, </span>2</span>, </span>2</span>, </span>3</span>, </span>3</span>, </span>3</span>, </span>4</span>, </span>4</span>, </span>4</span>]);
</span></code></pre>
This is not what we want: this has exactly the same semantics of
Stream::chain</code></a>, and will exhaust streams in-order. Ideally we'd like a random
mix of 1</code>s, 2</code>s, 3</code>s, and 4</code>s. In order to do that we need to randomly pick
between the items available from merge.</code> Unfortunately because we're merging
streams by chaining calls to merge</code> we can only pick between two streams at a
time 7</a></sup>. This causes the chance we'll pick an item from a stream to be skewed. For
the first 3 items in the stream, what we want is for each number to have exactly
a 25%</code> chance to be picked. But instead the chance we'll get a number is
skewed as follows:</p>

number 1</code>: 50% chance</li>
number 2</code>: 25% chance</li>
number 3</code>: 12.5% chance</li>
number 4</code>: 12.5% chance</li>
</ul>
The way to fix this is by making merge</code> aware of all</em> streams we're merging,
so we can pick fairly between them. We could do this either by creating a
merge!</code> macro, or by doing what we chose to do in futures-concurrency</code></a>:
implement a trait for tuples [^merge-macro]. Here's what using that looks like today:</p>
^{7</sup>
The reason for this is that when we call a.merge(b)</code> the merge</code>
function only knows about a</code> and b</code>. If b</code> holds the stream produced by
c.merge(d)</code>, then the probabilities divided between a</code>, b</code>, c</code> are now
skewed. This is really annoying, and the only way we can resolve this is if we know
how many items we need to choose from ahead of time</em>.</p>
</div>}let</span> a = stream::repeat(</span>1</span>).</span>take</span>(</span>3</span>);
</span>let</span> b = stream::repeat(</span>2</span>).</span>take</span>(</span>3</span>);
</span>let</span> c = stream::repeat(</span>3</span>).</span>take</span>(</span>3</span>);
</span>let</span> d = stream::repeat(</span>4</span>).</span>take</span>(</span>3</span>);
</span>
</span>let</span> s = (a, b, c, d).</span>merge</span>();
</span></code></pre>
This will gives us exactly the probability we're looking for, ensuring one
stream will not consistently be exhausted before another:</p>

number 1</code>: 25% chance</li>
number 2</code>: 25% chance</li>
number 3</code>: 25% chance</li>
number 4</code>: 25% chance</li>
</ul>
In past post's we've discussed the arity of the join</code>
APIs</a> and its API
consequences at length. But we haven't done so considering fairness</em>
specifically. For the join</code> family of APIs, it's generally an all-or-nothing
approach, so fairness is as not much of an issue 8</a></sup>. But for race</code>, which only
returns a single item, it might be. And so the same arguments we're making here
in favor of a trait-based approach for merge</code> apply to race</code> as well.</p>
^{8</sup>
Future::try_join</code> can return a single item when handling an
error, so fairness could</em> be an issue. So it's not an exact science. But I'm
not sure how important that is though. Maybe we just should account for it?</p>
</div>}Shiny Future</h2>
Thinking about a future where we introduce Stream::merge</code> (or more likely</a>:
AsyncIterator::merge</code>) into the stdlib paints a wholly different picture. It
already works well as-is. But as we add more features to the language that we
want anyway, using Stream::merge</code> would in turn become nicer to use as well.
So let's have fun shall we. What if Rust had all</em> the language features, what could our code look like then?:</p>

async iteration so we can write async for</code> loops</li>
match</code> shorthands so we can reduce some of the nesting</li>
Merge</code> implemented directly on tuples.</li>
an IntoAsyncIterator</code> trait implemented in all the same places as IntoIterator</code> is</li>
std::stream</code> renamed to std::async_iter</code></li>
</ul>
use </span>std::async_iter;
</span>
</span>// The shared output type
</span>enum </span>Message {
</span>    Num(</span>u8</span>),
</span>    Text(&</span>'static str</span>),
</span>}
</span>
</span>// Create our streams and map them to the shared output type.
</span>let</span> a = [</span>1</span>, </span>2</span>, </span>3</span>].</span>into_async_iter</span>().</span>map</span>(Message::Num);
</span>let</span> b = ["</span>chashu</span>", "</span>nori</span>", "</span>lily</span>"].</span>into_async_iter</span>().</span>map</span>(Message::Text);
</span>let</span> c = async_iter::once(async { "</span>hello</span>" }).</span>map</span>(Message::Text);
</span>
</span>// Merge the streams, iterate over them, and handle the output sequentially.
</span>for</span> await msg in (a, b, c).</span>merge</span>() </span>match </span>{
</span>    Num(n) => println!("</span>received a number: </span>{n}</span>"),
</span>    Text(s) => println!("</span>received a string: </span>{s}</span>"),
</span>}
</span></code></pre>
So far these features seem somewhat plausible: these are all features that are
fairly straight-forward syntactic sugar for existing patterns. And importantly:
none of these features are specific to the operations we're doing here. Which
means that even if we're looking at possible new language features here, none of
these are specific to async concurrency. But what if we
really</em> took it further. Let's dream big for a moment here. What if Rust also had:</p>

"async overloading"</a> and an async overload for IntoIter</code></li>
async IntoIter</code> implemented for all T: Future</code></li>
anonymous enums</a></li>
type ascription in match statements</li>
</ul>
// Create our streams and map them to the shared output type.
</span>// `into_iter` here assumes "async overloading".
</span>let</span> a = [</span>1</span>, </span>2</span>, </span>3</span>];
</span>let</span> b = ["</span>chashu</span>", "</span>nori</span>", "</span>lily</span>"];
</span>let</span> c = async { "</span>hello</span>" };
</span>
</span>// Merge the streams, iterate over them, and handle the output sequentially.
</span>// `merge` expects `T: async IntoIter`, which all types implement, so it just works.
</span>// The type of `msg` here is `u8 | &str`. This is inferred from the input types
</span>// and represented in the match statement.
</span>for</span> await msg in (a, b, c).</span>merge</span>() </span>match </span>{
</span>    n: </span>u8 </span>=> println!("</span>received a number: </span>{n}</span>"),
</span>    s: &</span>str </span>=> println!("</span>received a string: </span>{s}</span>"),
</span>}
</span></code></pre>
I can't say for sure whether this is a good idea or not. We're actively investigating
if async overloading can work. And I have no clue if an async Iterator for T where T: Future</code> blanket impl is a good idea. But the point of this
example is to have fun with it. To go big on the "what if". Maybe none of these
features will ever land. But it's still a useful exercise to understand to
which degree ergonomics are inherent of the API, and to which degree they're
tied to other factors.</p>
The future of Rust is what we make of it, so who knows if any of this will ever
happen. But even if it doesn't: we've already shown that achieving this is
possible using today's Rust. What we're showing here is purely just to make it
so we need to write less code to achieve the same effect.</p>
Addendum: ergonomics explorations for operating exclusively on futures</h2>
Hi, editor Yosh here. I'm adding this section close to the time I'd hit publish
on the post. My co-worker nrc</a> helped review this post, and they asked me
whether I could show an example using futures exclusively.</p>
We could directly implement merge</code> on Future</code>, creating a Stream</code>, but that
wouldn't lend itself well to chaining multiple futures: a.merge(b).merge(c)</code>
first creates a stream from a.merge(b)</code>, then tries to merge future c</code> to it,
which won't work.</p>
Working around this problem requires implementing the Merge</code> trait for Future</code>
containers as well, which would not work on stable Rust today. We'd want to use
negative trait bounds to express the right bounds:</p>

impl [T; N] where N: IntoFuture + !IntoStream</code></li>
impl [T; N] where N: IntoStream + !IntoFuture</code></li>
</ul>
And even with these bounds, merging Stream</code> and Future</code> types would still</em>
mean we'd require casting all futures to streams first. We could overcome this
by having a third trait, IntoMergable</code> or something, and then making our
generic container impls rely on that:</p>

impl IntoMergable for T where T: IntoFuture + !IntoStream</code></li>
impl IntoMergable for T where T: IntoStream + !IntoFuture</code></li>
[T; N] where N: IntoMergable</code></li>
</ul>
But that doesn't seem great. In general I think it's worth assessing the
following considering ergonomics in this order:</p>

How often does merge</code>-ing futures come up in practice?</li>
Is that often enough that it would warrant changes to the language?</li>
Would impl IntoStream for T: IntoFuture</code> be a feasible?</li>
</ol>
As I mentioned in earlier examples, my intuition is that whenever we want to
merge</code> futures, we likely could solve the problem more elegantly using
task::spawn{,_local}</code>. Meaning if we design APIs such as
TaskGroup</code></a> well,
we could carry Stream::merge</code>-like semantics through this. In fact, that is
exactly what Swift's TaskGroup</code>
type</a> does.
It can asynchronously iterate out all results from the group, allowing the
caller to handle them one-by-one.</p>
So to summarize this side-adventure: if we're looking to improve the semantics
of using Stream::merge</code> using futures only, there are a fair number of
questions and avenues we should explore before we draw any definitive
conclusions.</p>
The async Rust concurrency tables: completed</h2>
We can now begin filling in the original async Rust concurrency table using
Stream::merge</code>. Because Stream::merge</code> enables on items to be operated one at
the time, it can both "continue on error" and "return early on error":</p>
let</span> s = a.</span>merge</span>(b);
</span>
</span>// continue on error
</span>while let </span>Some(item) in s {
</span>    </span>// do something with `item`.
</span>}
</span>
</span>// return early on error
</span>while let </span>Some(item) in s {
</span>    </span>let</span> item = item?;
</span>    </span>// do something with `item`.
</span>}
</span></code></pre>
This means Stream::merge</code> allows you to choose from either short-circuiting behavior:</p>
</th> Wait for all outputs</strong></th> Wait for first output</strong></th> Handle output one-by-one</strong></th></tr></thead>

Continue on error</strong></td> Future::join</code></td> Future::try_race</code></td> Stream::merge</code></td></tr>
Return early on error</strong></td> Future::try_join</code></td> Future::race</code></td> Stream::merge</code></td></tr>
</tbody></table>
Regarding our second table, Stream::merge</code> enables the concurrency mode where
we "handle all items" and "start responding on the first item". This is the
top-left of the table. But what about the case where we're only interested in
the last item, and want to discard the rest? It turns out that that's a subset
of the functionality exposed by Stream::merge</code>, which we can get by using
Stream::last</code></a>:</p>
</th> Handle all items</strong></th> Discard some items</strong></th></tr></thead>

Response starts on first item</strong></td> Stream::merge</code></td> Future::race</code></td></tr>
Response starts on last item</strong></td> Future::join</code></td> Stream::merge</code> + Stream::last</code></td></tr>
</tbody></table>
Seeing this table as it is, you might wonder what the difference is between e.g.
Future::join</code> and Stream::collect</code>. Or converting futures into streams and
then breaking on the first item. You'd be right: in theory we could model
everything using streams and call it a day. But in practice it wouldn't feel good.</p>
The reason why Stream::merge</code> is the right answer for combining N futures is
because it allows us to express the time aspect of the operation</strong>. We don't
want all or nothing. We want items, in any order, as soon as they're made
available to us. Streams are "data over time". And even if the data is
originally held in futures, the time aspect can only be expressed using
streams.</strong></p>
In contrast, the Future::join</code> and Future::race</code> family of operations do not
rely on the same time aspect. join</code> yields all items.
race</code> just yields the one. So converting futures via streams back to singular
items is an indirect mapping which ends up feeling clunky to author. And worse:
it creates opportunities to introduce bugs. Future::join</code> and Future::race</code>
may be subsets of Stream::merge</code>, but they are unambiguous in what they
express which makes them a useful abstraction.</p>
Overall this
We might still want to change the names of the concurrency combinators later on.
But the core functionality, shape, and usage all feels stable and consistent.</p>
Conclusion</h2>
In this post we've looked at Rust's 5th mode of concurrency: "Await multiple
futures concurrently and operate on them as soon as they're ready."
And in passing also covered the 6th mode of concurrency:
"await multiple futures concurrently and discard all values but the last". We've
shown how this can be handled using the futures::select!</code> macro. How this can
better be handled using Stream::merge</code>. And talked about the various
considerations and future directions we have for this design.</p>
We've also done a real-world case analysis of an accidental mis-use of the
select!</code> macro, and shown how Stream::merge</code> would've prevented the same issue
from happening by making the failure case unrepresentable. Overall
Stream::merge</code> presents itself as a viable alternative to select!</code> for the
"handle one future at a time without ever discarding any" model of concurrency.</p>
Overall I'm incredibly pleased with the progress we've made with this series.
After nearly three years of work we finally have documented all modes of
concurrency for async Rust, and their corresponding interfaces. We can now start
experimenting with naming, polish, documentation, and then possibly (finally!)
start bringing some of these things into mainline Rust via RFCs. Things will
likely change between now and then, but it's good to finally stop needing to
think about semantics, and being able to focus more on ergonomics. It's been a
slow process, but I'm incredibly happy with the outcome. And thank you for
following along over all of these years!</p>
If you liked this post and would like to see my cats, you can follow me on
Twitter</a>.</p>
Thanks to: Ryan Levick</a>, Nick Cameron</a> and Irina Shestak</a> for reviewing drafts of this post.</em></p>
Appendix: Async Rust Concurrency Operations</h2>
These are all the concurrency operations we want to be able to express in async
Rust.</p>
1. Concurrency adapters by fallibility mode and output handling</h4>
</th> Wait for all outputs</strong></th> Wait for first output</strong></th> Handle output one-by-one</strong></th></tr></thead>

Continue on error</strong></td> Future::join</code></td> Future::try_race</code></td> Stream::merge</code></td></tr>
Return early on error</strong></td> Future::try_join</code></td> Future::race</code></td> Stream::merge</code></td></tr>
</tbody></table>
2. Concurrency adapters by response start and output handling</h4>
</th> Handle all items</strong></th> Discard some items</strong></th></tr></thead>

Response starts on first item</strong></td> Stream::merge</code></td> Future::race</code></td></tr>
Response starts on last item</strong></td> Future::join</code></td> Stream::merge</code> + Stream::last</code></td></tr>
</tbody></table>

Uninit Read/Write
Tue, 07 Dec 2021 00:00:00 +0000
The AsyncRead</code> and AsyncWrite</code> traits are async versions of the Read</code> and
Write</code> traits in Rust. They're core to async Rust, providing the interface to
read and write bytes from for example the filesystem and network. But both the
async and non-async variants have an open issue: how can we use these traits to
write data into uninitialized memory?</p>
In this post we look at the problem of reading into unitialized memory using
Read</code>/Write</code>, and at possible solution we could introduce to solve this.</p>
Showing the problem</h2>
Both async and non-async Rust share the same issue for both Read</code> and Write</code>
traits. So let's take use the non-async Read</code> trait as our example. It's
defined as follows:</p>
pub trait </span>Read {
</span>    </span>fn </span>read</span>(&</span>mut </span>self</span>, </span>buf</span>: &</span>mut</span> [</span>u8</span>]) -> Result<</span>usize</span>>;
</span>}
</span></code></pre>
Calling Read::read</code> will read bytes into a mutable slice of memory, and return
an io::Result</code> containing either the number of bytes read, or an error. Usage
typically is something like this:</p>
// open a file and init the buffer
</span>let mut</span> f = File::open("</span>foo.txt</span>")?;
</span>let mut</span> buffer = vec![</span>0</span>; </span>1024</span>];
</span>
</span>// read up to 1024 bytes
</span>let</span> read = f.</span>read</span>(&</span>mut</span> buffer)?;
</span>let</span> data = buffer[..read];
</span></code></pre>
This will read 1024 bytes of the file into the buffer. But it comes with a slight
inefficiency: we're writing zeroes into the buffer, and then immediately writing
more data into the buffer after that. This means we're doing a double-write,
which is not as efficient as it could be</strong>
(report</a>).
Ideally we'd be able to do the following:</p>
// open a file and init the buffer
</span>let mut</span> f = File::open("</span>foo.txt</span>")?;
</span>let mut</span> buf = Vec::with_capacity(</span>1024</span>); </span>// reserve capacity, but don't initialize
</span>
</span>// read up to 1024 bytes
</span>let</span> read = f.</span>read</span>(&</span>mut</span> buf)?;
</span>let</span> data = buffer[..read];
</span></code></pre>
But this doesn't work. Even if the capacity</em> is 1024, because we
haven't initialized the vector it's as if we passed an empty slice, and the
following assertion will always hold:</p>
assert_eq!(data.</span>len</span>(), </span>0</span>);
</span></code></pre>
And this is the problem this post is about: our slices can't have uninitialized
memory, and our vectors which can</em> have uninitialized memory are always
dereferenced into slices first. In order to support writing into uninitialized
memory over Read</code> bounds we need to resolve this.</p>
Solution 1: unitialized slices</h2>
The current thinking is that we can solve this issue by introducing "slices of
unitialized memory" and extending the IO traits with methods dedicated to taking
these slices. This was proposed and subsequently merged as part of RFC
2930: read_buf</a>
(tracking issue</a>).</p>
The RFC is worth a read if you want a closer look at the problem space. But the
gist of the solution it proposes is as follows:</p>

Add a new ReadBuf</code> type which wraps [MaybeUninit::<u8>::uninit(); N];</code></li>
Add an optional read_buf</code> method which takes &mut ReadBuf<'_></code>
instead of &mut [u8]</code>.</li>
Users who want to read into uninitialized memory can implement and use
read_buf</code>.</li>
</ul>
The RFC provides the following usage example:</p>
let mut</span> buf = [MaybeUninit::<</span>u8</span>>::uninit(); </span>8192</span>];
</span>let mut</span> buf = ReadBuf::uninit(&</span>mut</span> buf);
</span>
</span>loop </span>{
</span>    some_reader.</span>read_buf</span>(&</span>mut</span> buf)?;
</span>    </span>if</span> buf.</span>filled</span>().</span>is_empty</span>() {
</span>        </span>break</span>;
</span>    }
</span>    </span>process_data</span>(buf.</span>filled</span>());
</span>    buf.</span>clear</span>();
</span>}
</span></code></pre>
With the following definition for Read::read_buf</code> provided by default, intended
to be modified by implementers:</p>
impl </span>Read </span>for </span>MyReader {
</span>    </span>fn </span>read_buf</span>(&</span>mut </span>self</span>, </span>buf</span>: &</span>mut </span>ReadBuf<'_>) -> io::Result<()> { 
</span>        </span>let</span> n = </span>self</span>.</span>read</span>(buf.</span>initialize_unfilled</span>())?;
</span>        buf.</span>add_filled</span>(n);
</span>        Ok(())
</span>    }
</span>}
</span></code></pre>
This achieves the goal it sets out to do: we successfully can read data into
uninitialized memory. However for users it requires a little more work to setup:</p>
// Read into uninit memory.
</span>let mut</span> buf = [MaybeUninit::<</span>u8</span>>::uninit(); </span>1024</span>];
</span>let mut</span> buf = ReadBuf::uninit(&</span>mut</span> buf);
</span>file.</span>read_buf</span>(&</span>mut</span> buf)?;
</span>let</span> data = buf.</span>filled</span>();
</span>
</span>// Read into init memory
</span>let mut</span> buf = [</span>0</span>; </span>1024</span>];
</span>let</span> read = file.</span>read</span>(&</span>mut</span> buf)?;
</span>let</span> data = buf[..read];
</span></code></pre>
Overall this is not too bad, and gets us in the right direction. In order to
write bytes to the heap instead of the stack I believe you can wrap the buf in a
Box::new</code> 1</a></sup> 2</a></sup>. The main downsides are that it requires using
a dedicated type just for this purpose, and it's slightly more verbose.</p>
^1</sup>I believe with placement new (box</code>
keyword</a>) this might save
yet another copy.</p>
</div>
^2</sup>Or as maybewaffle pointed
out</a>: this could be
used with Vec::spare_capacity_mut</code> as well.</p>
</div>
Solution 2: uninitialized vectors</h2>
Since August 2020, the
Vec::spare_capacity_mut</code></a>
method has been available on nightly. This returns the remaining spare capacity
of the vector as a slice of MaybeUninit<T></code>. The docs show the following usage
example:</p>
let mut</span> v = Vec::with_capacity(</span>10</span>);
</span>let</span> uninit = v.</span>spare_capacity_mut</span>();
</span>
</span>uninit[</span>0</span>].</span>write</span>(</span>0</span>);
</span>uninit[</span>1</span>].</span>write</span>(</span>1</span>);
</span>uninit[</span>2</span>].</span>write</span>(</span>2</span>);
</span>
</span>// Mark the first 3 elements of the vector as being initialized.
</span>unsafe </span>{
</span>    v.</span>set_len</span>(</span>3</span>);
</span>}
</span>assert_eq!(&v, &[</span>0</span>, </span>1</span>, </span>2</span>]);
</span></code></pre>
This API fills much the same role as the ReadBuf</code> type did in the first
solution, but without requiring a new type to be introduced. Usage of
uninitialized vectors becomes a lot closer to the current Read::read</code> behavior:</p>
// Read into uninit memory.
</span>let mut</span> buf = Vec::with_capacity(</span>1024</span>);
</span>let</span> read = file.</span>read_buf</span>(&</span>mut</span> buf)?;
</span>let</span> data = buf[..read];
</span>
</span>// Read into init memory
</span>let mut</span> buf = [</span>0</span>; </span>1024</span>];
</span>let</span> read = file.</span>read</span>(&</span>mut</span> buf)?;
</span>let</span> data = buf[..read];
</span></code></pre>
Implementers of "leaf" IO types (e.g. File</code>, etc.) could use this method to
implement a reader which reads directly into uninitialized memory. The default
implementation of read_buf</code> could become something like this:</p>
impl </span>Read </span>for </span>MyReader {
</span>    </span>fn </span>read_buf</span>(&</span>mut </span>self</span>, </span>buf</span>: &</span>mut </span>Vec<</span>u8</span>>) -> io::Result<</span>usize</span>> { 
</span>        </span>// zero-fill the unintialized memory
</span>        </span>let</span> uninit = buf.</span>spare_capacity_mut</span>();
</span>        mem::swap(uninit, &</span>mut </span>MaybeUninit::zeroed());
</span>        </span>unsafe </span>{ buf.</span>set_len</span>(buf.</span>capacity</span>()) };
</span>
</span>        </span>// read the data into the buffer
</span>        </span>self</span>.</span>read</span>(buf)
</span>    }
</span>}
</span></code></pre>
It's worth pointing out that the name read_buf</code> at this point doesn't convey
the intent particularly well anymore. A name such as read_to_capacity</code> might
more closely match our intent. To my knowledge: "Read into the uninitialized
bytes of a Vec</code>, but don't grow beyond it"</em>, doesn't have any precedent in the
stdlib. The closest I can think of is Vec::fill</code></a>,
but it's not quite it. If we choose to use this API, method naming and
documentation is something we should take a more closer look at.</p>
Even reading into the stack becomes possible if we use
Vec::with_capacity_in</code></a>
backed by a stack allocator. The ergonomics of that aren't ideal yet, but that's
expected to significantly improve over time 3</a></sup>.</p>
^{3</sup>
I know folks are actively looking at the ergonomics of this, and am
excited for some of the designs I've seen.</p>
</div>
As we mentioned, the main benefit of this approach is that it doesn't introduce
a new type and as such maps nicely to existing patterns. Compared to the first
solution we have better ergonomics when reading into the heap, and slightly
worse ergonomics when reading into the stack. Overall I think this approach is
promising, and I prefer it over the first solution.</p>}Solution 3: specialization</h2>
Disclaimer: I'm by no stretch an authority on dynamic dispatch or
specialization. It might be that I got details wrong, or misunderstood
limitations. This section is "best effort", and am happy to update it if turns
out I got something wrong.</em></p>
Going back to our earlier example, an ideal solution would be if instead of
defining a new read_buf</code> methods we could keep using the read</code> method instead:</p>
// Read into uninit memory.
</span>let mut</span> buf = Vec::with_capacity(</span>1024</span>);
</span>let</span> read = file.</span>read</span>(&</span>mut</span> buf)?; </span>// note: `read`, not `read_buf`
</span>let</span> data = buf[..read];
</span></code></pre>
The way to do this in Rust would be using specialization</em>. Specialization is
still considered "unstable", and has not yet been fully formed. So don't expect
the code below to work anytime soon, if ever. But if we squint a little, we
could imagine a trait could be defined as follows:</p>
pub trait </span>Read {
</span>    </span>// The "default" implementation.
</span>    default </span>fn </span>read</span>(&</span>mut </span>self</span>, </span>buf</span>: &</span>mut</span> [</span>u8</span>]) -> Result<</span>usize</span>>;
</span>
</span>    </span>// Implementation specialized for `&mut Vec<u8>`
</span>    </span>fn </span>read</span>(&</span>mut </span>self</span>, </span>buf</span>: &</span>mut </span>Vec<</span>u8</span>>) -> Result<</span>usize</span>> {
</span>        </span>// zero-fill the unintialized memory
</span>        </span>let</span> uninit = buf.</span>spare_capacity_mut</span>();
</span>        mem::swap(uninit, &</span>mut </span>MaybeUninit::zeroed());
</span>        </span>unsafe </span>{ buf.</span>set_len</span>(buf.</span>capacity</span>()) };
</span>
</span>        </span>// read the data into the buffer
</span>        </span>self</span>.</span>read</span>(buf.</span>as_slice</span>())
</span>    }
</span>}
</span></code></pre>
In this example we define the trait Read</code> with a "default" implementation for
&mut [u8]</code>, and a built-in specialization for &mut Vec<u8></code>. The
specialization needs to be built directly into the trait definition, to ensure
that &mut Vec<u8></code> will never default to being interpreted as an zero-length
slice. Trait implementers would be expected to provide their own specialization
for &mut Vec<u8></code>, to make use of the ability to write directly into
uninitialized memory.</p>
Another thing that's unclear about specializations is the interactions with
semver. In our example we're now using Vec::capacity</code> rather than Vec::len</code> to
determine how many bytes to write into the vector. All observable changes in
behavior have the potential to be breaking</a>, so any
modification to this behavior should always use crater to determine impact. But
it doesn't seem like we would break any intended use of the APIs.</p>
At first glance this interface might seem ideal. It's the same
interface that just works</em> depending on what's passed. But as remarked in
RFC 2930: read_buf</a>,
this approach has issues:</p>

It must be compatible with dyn Read. Trait objects are used pervasively in
IO code, so a solution can't depend on monomorphization or specialization.</li>
</ol>
</blockquote>
Further elaboration on this point exists in this
document</a>
and
this interview</a>.
My interpretation of these materials is: "Specialization is not dyn safe, so if
we add a specialization dyn Read</code> would no longer be possible". That would mean
the if Read</code> specialized as we showed, the following code would fail to
compile:</p>
// This alias would cause an error because `Read` is not dyn safe.
</span>type </span>MyType = Box<dyn Read>;
</span></code></pre>
Intuitively I would've expected this to be possible if we wanted it to, even if
it's currently not implemented in the compiler. But my understanding of dyn</code> is
limited compared to the folks who've written the docs, so I trust there are good
reasons why they marked it as a non-option.</p>
That said though; the constraints around dyn</code> in Rust are currently being
reworked in order to enable things like an AsyncIterator</code> with async fn next</code></a>
to work. Which makes me wonder: is the interaction between dyn</code> and
specialization something which can be reworked as well?</p>
I asked my colleague Sy</a> whether combining
dynamic dispatch with specialization is possible in C++, and it appears it is!
They walked me through this C++ example</a> which
combines a dynamic dispatch based on the class it's called on and whether a
vec</code> or span</code> (slice) is passed. Even though the code is very different from
Rust, the functionality matches how we would expect it to work.</p>
If we think about the wider implications of this, it seems not great</em> if
specializations and dynamic dispatch would inherently remain mutually exclusive.
Both are incredibly useful, and if they can't be used together that seems
incredibly limiting.</p>
Summary</h2>
In this post we've shown three examples for reading into uninitialized memory in
Rust. As I mentioned throughout the post: it'd be great if we could make reading
into unitialized memory as close as possible to reading into initialized memory.
And I think in particular the second, and third examples have a lot of potential.</p>
I was motivated to write this post after conversations with
nrc</a> last week. I remembered reading
Amanieu's</a> comment about spare_capacity_mut</code> on
the tracking issue for RFC 2930</a>,
and started thinking of what it could look like if we used that instead of
ReadBuf</code>.</p>
In terms of timelines, I don't believe we're under great pressure to land the
features described in this post into the stdlib. The performance benefits can be
significant under certain workloads; but as we're collectively moving towards
completion-based IO, traits such as
AsyncBufRead</code></a>
will become more relevant</a>.
Because these traits manage buffers internally, uninitialized memory never needs
to cross trait boundaries and this post doesn't apply.</p>
I hope I was able to provide some insight on how we can enable reading into
uninitialized memory for (async) IO traits. If you liked this post and would
like to see my cats, you can follow me on Twitter</a>.</p>
Thanks to Sy Brand</a> for showing me how
dynamic dispatch and specialization interact in C++, and
nrc</a> for helping review this post.</em></p>

Async Cancellation I
Wed, 10 Nov 2021 00:00:00 +0000
Sometimes we start things but decide midway through that we would prefer to
rather not be doing them. That process is sometimes referred to as cancellation.
Say we accidentally clicked "download" on a large file in the browser. We should
have a way to tell the computer to stop downloading it.</p>
When the async foundations WG was researching user stories</a>
earlier this year, async cancellation came up repeatedly. It's one of those
things that's important to have, but can be tricky to reason about. This is not
helped by the fact that little has been written about it, so I figured I might
try to make a dent in that by writing a deep-dive on the topic.</p>
In this post we'll look at async Rust's async primitives, and cover how
cancellation works for those primitives today. We'll then proceed to look at
ways in which we can ensure we do not end up with dangling resources. And
finally we'll take a look at what the current direction of async Rust means for
async cancellation. Sounds like a plan? Good, let's dive in!</p>
Tasks and Futures</h2>
For the purpose of this post we need to distinguish between two types of async
primitives in Rust: futures and tasks 1</a></sup>.</p>
^{1</sup>
Arguably "Stream" / "AsyncIterator" is a third async primitive, but
everything we say about the Future</code> type applies to Stream</code> as well, so we're
considering those the same in this post.</p>
</div>}

Futures</strong> represent the core building block of async Rust, and exist as
part of the Rust language and stdlib. A future is something that when .await</code>ed
becomes a value later</em>. By design it does nothing unless .await</code>ed, and must
be driven to completion by some other, external loop (example of such a
loop</a>).</p>
</li>

Tasks</strong> are not yet part of the Rust language or stdlib, and represent a
managed 2</a></sup> piece of async execution backed by an asynchronous
runtime 3</a></sup>. Tasks often allow for async code to become
multi-threaded: it's common for executors to schedule tasks across multiple
threads, and even move them between threads after they've started executing. A
task does not need to be .await</code>ed before it starts executing, and the future
it returns is just a way to get the return value of the task once it finishes.</p>
</li>
</ul>
^{2</sup>
"managed" is used throughout this post as: "is scheduled on a
runtime". This typically comes with the requirement futures are 'static</code> (the
future does not hold any borrows), and when scheduled on a multi-threaded
runtime often, but not always, requires futures be Send</code> as well (the future is
thread-safe).</p>
</div>}^{3</sup>
You can think of Tasks as being something akin to
lightweight threads, managed by your program rather than you operating system.</p>
</div>}Many languages, including JavaScript, use equivalents to Rust tasks rather than
Rust futures as their core building blocks 4</a></sup>. This is convenient because
only a single type of async building block is exposed, and the language runtime
optimizers can figure out how to speed things up if needed. But in Rust we
unfortunately can't rely on that, so we manually distinguish between unmanaged
(futures) and managed (tasks) primitives.</p>
^4</sup>JavaScript's counterpart to Rust's tasks is
Promise</code></a>.
It starts being executed the moment it's instantiated, rather than the moment
it's await</code>ed. My understanding is that C#'s
Task<T></code></a>
works much the same way.</p>
</div>
Cancelling Futures</h2>
Cancellation allows for a future to stop doing work early, when we know we're no
longer interested in its result. Futures in Rust can be cancelled</em> at one of
two points. For simplicity most our examples in this post are going to use
sleeping and printing operations; where in the real world we'd probably be
talking about file/network operations and data processing instead.</p>
1. Cancel a future before it starts executing</strong></p>
Here we create a drop guard, pass it to an async function which returns a
future, and then drop the future before we .await</code> it:</p>
use </span>std::time::Duration;
</span>
</span>struct </span>Guard;
</span>impl </span>Drop </span>for </span>Guard {
</span>    </span>fn </span>drop</span>(&</span>mut </span>self</span>) {
</span>        println!("</span>2</span>");
</span>    }
</span>}
</span>
</span>async </span>fn </span>foo</span>(</span>guard</span>: Guard) {
</span>    println!("</span>3</span>");
</span>    task::sleep(Duration::from_secs(</span>1</span>)).await;
</span>    println!("</span>4</span>");
</span>}
</span>
</span>fn </span>main</span>() {
</span>    println!("</span>1</span>");
</span>    </span>let</span> guard = Guard {};
</span>    </span>let</span> fut = </span>foo</span>(guard);
</span>    </span>drop</span>(fut);
</span>    println!("</span>done</span>");
</span>}
</span></code></pre>
This prints:</p>
> 1
</span>> 2
</span>> done
</span></code></pre>
Our Guard</code> type here will print when its
destructor (Drop</code>)</a> is run.
We never actually execute the future, but the destructor is still run because we
passed the value to the async function. This means the first cancellation point
of any future is immediately after instantiation before the async functions's
body has run. Meaning not all cancellation points are necessarily demarked by
.await</code>.</p>
2. Cancel a future at an await point</strong></p>
Here we create a future, poll it exactly once, and then drop it:</p>
use </span>std::{ptr, task};
</span>use </span>async_std::task;
</span>use </span>std::time::Duration;
</span>
</span>async </span>fn </span>foo</span>() {
</span>    println!("</span>2</span>");
</span>    task::sleep(Duration::from_secs(</span>1</span>)).await;
</span>    println!("</span>3</span>");
</span>}
</span>
</span>let mut</span> fut = Box::pin(</span>foo</span>());
</span>let mut</span> cx = </span>empty_cx</span>();
</span>
</span>println!("</span>1</span>");
</span>assert!(fut.</span>as_mut</span>().</span>poll</span>(&</span>mut</span> cx).</span>is_pending</span>());
</span>drop</span>(fut);
</span>println!("</span>done</span>");
</span>
</span>/// Create an empty "Waker" callback, wrapped in a "Context" structure.
</span>/// How this works is not particularly important for the rest of this post.
</span>fn </span>empty_cx</span>() -> task::Context { ... }
</span></code></pre>
> 1
</span>> 2
</span>> done
</span></code></pre>
Fundamentally, you can think of .await</code> as marking a point where
cancellation may occur.</strong> Where keywords like return</code> and ?</code> mark points where
the function may return a value, .await</code> marks a location where the function's
caller may decide the function should not run any further. But importantly: in
all cases destructors will be run, allowing resources to be cleaned up.</p>
Futures cannot be cancelled in between .await</code> calls, or after the last
.await</code> call. We also don't yet have a design for "async Drop</code>" so we also
can't say anything meaningful yet about how how that will interact with
cancellation.</p>
Cancelling Tasks</h2>
Because tasks aren't standardized in Rust yet, cancellation of tasks isn't
either. And unsurprisingly different runtimes have different ideas on how to
cancel a task. Both async-std</code> and tokio</code> share a similar task model.
I'm most comfortable with async-std</code> 5</a></sup>, so let's use that as an
example:</p>
^{5</sup>
Probably because I co-authored the project.</p>
</div>}use </span>async_std::task;
</span>
</span>let</span> handle = task::spawn(async {
</span>    task::sleep(Duration::from_secs(</span>1</span>)).await;
</span>    println!("</span>2</span>");
</span>});
</span>
</span>println!("</span>1</span>");
</span>drop</span>(handle);     </span>// Drop the task handle.
</span>task::sleep(Duration::from_secs(</span>2</span>)).await;
</span>println!("</span>done</span>");
</span></code></pre>
> 1
</span>> 2
</span>> done
</span></code></pre>
Here we dropped the handle, but the task continued to run in the background.
This is because the async-std</code> runtime uses "detach on drop" semantics for
tasks. This is the same in the tokio</code> runtime. In order to cancel a task, we
need to manually call a method on the handle. For async-std</code> this is
JoinHandle::cancel</code></a>,
and for tokio</code> this is
JoinHandle::abort</code></a>:</p>
use </span>async_std::task;
</span>
</span>let</span> handle = task::spawn(async {
</span>    task::sleep(Duration::from_secs(</span>1</span>)).await;
</span>    println!("</span>2</span>");
</span>});
</span>
</span>println!("</span>1</span>");
</span>handle.</span>cancel</span>().await;    </span>// Cancel the task handle
</span>task::sleep(Duration::from_secs(</span>2</span>)).await;
</span>println!("</span>done</span>");
</span></code></pre>
> 1
</span>> done
</span></code></pre>
We can see that if we call JoinHandle::cancel</code>, the task is cancelled at that
point, and the number 2</code> is no longer printed.</p>
Propagating cancellation for futures</h2>
In Async Rust cancellation automatically propagates for futures. When we stop
polling a future, all futures contained within will in turn also stop making
forward progress. And all
destructors</a> within them
will be run. In this example we'll be using the
FutureExt::timeout</code></a>
function from async-std</code>, which returns a Result<T, TimeoutError></code>.</p>
use </span>async_std::prelude::*;
</span>use </span>async_std::task;
</span>use </span>std::time::Duration;
</span>
</span>async </span>fn </span>foo</span>() {
</span>    println!("</span>2</span>");
</span>    </span>bar</span>().</span>timeout</span>(Duration::from_secs(</span>3</span>)).await;
</span>    println!("</span>5</span>");
</span>}
</span>
</span>async </span>fn </span>bar</span>() {
</span>    println!("</span>3</span>");
</span>    task::sleep(Duration::from_secs(</span>2</span>)).await;
</span>    println!("</span>4</span>");
</span>}
</span>
</span>println!("</span>1</span>");
</span>foo</span>().</span>timeout</span>(Duration::from_secs(</span>1</span>)).await;
</span>println!("</span>done</span>");
</span></code></pre>
> 1
</span>> 2
</span>> 3
</span>> done    # `4` and `5` are never printed
</span></code></pre>
The tree of futures above can be expressed as the following graph:</p>
main
</span>  -> foo (times out after 1 sec)
</span>    -> bar (times out after 3 secs)
</span>      -> task::sleep (wait for 2 secs)
</span></code></pre>
Because foo</code> is timed out and dropped before bar</code> has the chance to complete,
not all numbers get to be printed. bar</code> is dropped before it has the opportunity
to complete, and all of its resources are cleaned up when its destructors run.</p>
This means that in order to cancel a chain of future, all we need to do is to
drop it</em>, and all resources will in turn be cleaned up. Whether we're dropping
a future by hand,
calling a timeout method</a>
on a future, or
racing multiple futures</a>
— cancellation will propagate, and resources will be cleaned up.</p>
Propagating cancellation for tasks</h2>
As we showed earlier, simply dropping a task handle is not enough to cancel a
task in most runtimes. We need to explicitly invoke a cancellation method to
cancel the task. This means tasks aren't automatically propagated.</p>
In hindsight this was probably a mistake. More specifically, this was likely
my</em> mistake. async-std</code>'s JoinHandle</code> was the first in the ecosystem,
and I pushed that we should model it using the "detach-on-drop" behavior
directly after std::thread::JoinHandle</code>. What I didn't account for though,
is that threads cannot be cancelled externally: std::thread::JoinHandle</code>
doesn't, and may likely never have a cancel</code> method 6</a></sup>.</p>
^{6</sup>
The way a thread can be cancelled from the outside is
by passing a channel to the thread, and cancelling when a message is received.
Unlike async Rust, threads don't have .await</code> points which act as natural
stopping points. So instead threads have to opt-in to cancellation by manually
listening for signals.</p>
</div>
Failing to propagate cancellation means that if we cancel a tree of work, we
may end up with dangling tasks</strong> that continue when we really don't want to.
Rather than having a cancel</code> method which allows us to manually opt-in to
cancellation propatation (more on that later), cancellation propagation should
be opt-out by default.</p>}Luckily we don't need to guess what a runtime with "cancellation propagation is
opt-out" behavior would look like. The
async-task</code></a>
executor, and in turn smol</code></a> runtime do
exactly that.</p>
smol::block_on(async {
</span>    println!("</span>1</span>");
</span>    </span>let</span> task = smol::spawn(async {
</span>        println!("</span>2</span>");
</span>    });
</span>    </span>drop</span>(task);
</span>    println!("</span>done</span>")
</span>});
</span></code></pre>
> 1
</span>> done
</span></code></pre>
Patching cancellation propagation</h2>
Unfortunately only smol</code> propagates cancellation across tasks, and it would be
a breaking change to modify the cancellation propagation behavior of
JoinHandle</code> in other runtimes.</p>
Users of runtimes can still ensure that cancellation will always correctly be
propagated by creating a custom spawn</code> function that contains a drop guard like
so:</p>
use </span>tokio::task;
</span>use </span>core::future::Future;
</span>use </span>core::pin::Pin;
</span>use </span>core::task::{Context, Poll};
</span>
</span>/// Spawn a new tokio Task and cancel it on drop.
</span>pub fn </span>spawn</span><T>(</span>future</span>: T) -> Wrapper<</span>T::</span>Output>
</span>where
</span>    T: Future + Send + </span>'static</span>,
</span>    </span>T::</span>Output: Send + </span>'static</span>,
</span>{
</span>    Wrapper(task::spawn(future))
</span>}
</span>
</span>/// Cancels the wrapped tokio Task on Drop.
</span>pub struct </span>Wrapper<T>(task::JoinHandle<T>);
</span>
</span>impl</span><T> Future </span>for </span>Wrapper<T>{
</span>    </span>type </span>Output = Result<T, task::JoinError>;
</span>    </span>fn </span>poll</span>(</span>mut </span>self</span>: Pin<&</span>mut Self</span>>, </span>cx</span>: &</span>mut </span>Context<'_>) -> Poll<</span>Self::</span>Output> {
</span>        </span>unsafe </span>{ Pin::new_unchecked(&</span>mut </span>self</span>.</span>0</span>) }.</span>poll</span>(cx)
</span>    }
</span>}
</span>
</span>impl</span><T> Drop </span>for </span>Wrapper<T> {
</span>    </span>fn </span>drop</span>(&</span>mut </span>self</span>) {
</span>        </span>// do `let _ = self.0.cancel()` for `async_std::task::Task`
</span>        </span>self</span>.</span>0.</span>abort</span>();
</span>    }
</span>}
</span></code></pre>
This wrapper can be adapted to work for async-std</code> as well, and ensures that
cancellation propagates across task bounds.</p>
Structured concurrency</h2>
Given we're talking a lot about propagating cancellation and trees in this post,
we probably should mention the concept of "structured
concurrency"</a>.</p>
In order for async Rust to be structurally concurrent I think of tasks as having
the following requirements:</p>

Ensure 7</a></sup> sure child tasks don't outlive their parents.</li>
If we cancel a parent task the child task should be cancelled too.</li>
If a child task raises an error, the parent task should be able to act on it.</li>
</ol>
^{7</sup>
More on whether we can actually ensure this later on in this post.</p>
</div>
This structure makes it so errors don't accidentally go ignored, or we fail to
cancel tasks further down in the tree. It's the basis for effectively
implementing other mechanisms on top, such as retries, limits, supervisors, and
transactions.</p>}Cancelling a group of tasks</h2>
Cancellation becomes more difficult to implement when we're dealing with a
stream of tasks, each of which needs to be spawned on the runtime. Currently a
lot of async code just spawns tasks, detaches them, and logs if case of an
error:</p>
// The async-std "echo tcp server" example.
</span>use </span>async_std::io;
</span>use </span>async_std::net::{TcpListener, TcpStream};
</span>use </span>async_std::prelude::*;
</span>use </span>async_std::task;
</span>
</span>// Listen for new TCP connections on port 8080.
</span>let</span> listener = TcpListener::bind("</span>127.0.0.1:8080</span>").await?;
</span>
</span>// Iterate over the incoming connections, and move each task to a multi-threaded
</span>// runtime.
</span>let mut</span> incoming = listener.</span>incoming</span>();
</span>while let </span>Some(stream) = incoming.</span>next</span>().await {
</span>    task::spawn(async {
</span>        </span>// If an error occurs, log it to stderr.
</span>        </span>if let </span>Err(err) = </span>run</span>(stream).await {
</span>            eprintln!("</span>error: </span>{}</span>", err);
</span>        }
</span>    });
</span>}
</span>
</span>// The main logic of our listen loop. This is a simple echo server.
</span>async </span>fn </span>run</span>(</span>stream</span>: io::Result<TcpStream>) -> io::Result<()> {
</span>    </span>let</span> stream = stream?;
</span>    </span>let </span>(reader, writer) = &</span>mut </span>(&stream, &stream);
</span>    io::copy(reader, writer).await?;
</span>    Ok(())
</span>}
</span></code></pre>
But if we're trying to do Structured
Concurrency</a>
correctly, we need to ensure cancellation propagates to those spawned tasks too.
In order to do so we need to introduce a new async primitive to Rust: the
TaskGroup</code>8</a></sup>.</p>
^8</sup>This terminology is borrowed from Swift, but is similar-ish to how
crossbeam-scope</code></a> works (n threads
managed by a central point). However unlike crossbeam-scope</code>, the name concerns
itself less with how the lifetimes flow, and more how you can reason about how
it should be used.</p>
</div>
To my knowledge no runtimes currently support this out of the box, but crates
for this exist on crates.io. One example of such a crate is
task-group</code></a>, authored by the Fastly WASM
group. Functionally it allows creating a group of tasks which act as a single
unit. If a task errors or panics, all other tasks are cancelled. And when the
TaskManager</code> is dropped, all currently running tasks are cancelled.</p>
Task grouping (including task scopes) is a topic that needs its own blog post.
But if you're looking to apply cancellation propagation to a stream of items, at
least now you know this is the primitive that enables that to work.</p>
halt-safety</h2>
So far we've talked a fair bit about cancellation in this post. Now that we know
that the existence of .await</code> means that our function might exit, it's natural
to ask: "If our Future's state machines can be cancelled at any state, how do
we ensure they function correctly?"</em></p>
When we're using futures created through async/.await</code>, cancellation at
.await</code> points will functionally act no different than early returns through
the try (?</code>) operator.</strong> In both cases function execution is halted,
destructors are run, and resources are cleaned up. Take the following example;</p>
// Regardless of where in the function we stop execution, destructors will be
</span>// run and resources will be cleaned up.
</span>async </span>fn </span>do_something</span>(</span>path</span>: PathBuf) -> io::Result<Output> {
</span>                                        </span>// 1. the future is not guaranteed to progress after instantiation
</span>    </span>let</span> file = fs::open(&path).await?;  </span>// 2. `.await` and 3. `?` can cause the function to halt
</span>    </span>let</span> res = </span>parse</span>(file).await;        </span>// 4. `.await` can the function to halt
</span>    res                                 </span>// 5. execution has finished, return a value
</span>}
</span></code></pre>
do_something</code> has 5 distinct points where it can halt execution and destructors
are run. It doesn't matter whether .await</code> triggers a cancellation, ?</code> returns
an error, or our function exits as expected. Our destructors don't need to care,
they only need to concern themselves with is ensuring they release whatever
resources they're holding onto 9</a></sup>.</p>
^{9</sup>
"Clean up resources regardless of what triggered the function to
halt" is what I'm referring to as "halt-safety". I'm not in love with the term,
but I needed to find a way to give a name to this group of mechanisms. I hope
the term is clear enough.</p>
</div>}Things change a bit when we talk about Futures created using manual
Poll</code></a> state machines.
Within a manual state machine we don't call .await</code>; instead we make forward
progress by manually calling Future::poll</code> or Stream::poll_next</code>. Similar when
dealing with async/.await</code> we want to ensure that destructors are run should
any of poll_*</code>, ?</code>, or return</code> run. Unlike async/.await</code> futures,
resiliency against our function halting isn't automatically provided for us.
Instead manually authored futures need to implement the resiliency that
async/.await</code> futures rely on.</strong></p>
I like to think about a futures call graph like a tree. There are the leaf
nodes in the tree, which need to guarantee cancellation is handled correctly.
And there are branch nodes in the tree, which are all created through
async/.await</code> and rely on the leaf nodes correctly implementing resource
cleanups.</p>
Kind of like how in non-async Rust we don't really need to think about releasing
file handles or memory since we rely on that just working out of the box. The
only time we really need to think about how to release resources, is when we're
implementing primitives like TcpStream</code> or Vec</code> ourselves.</p>
Should tasks be detachable?</h2>
We've talked quite a bit about the importance of cancellation propagation for
tasks, perhaps to the point that you might now be wondering whether tasks should
be detachable at all. Unfortunately destructors in Rust are not guaranteed to
run</a>, so we can't can't
actually stop people from passing JoinHandle</code>s to mem::forget</code> in order to
detach their tasks.</p>
That does not mean that we should make detaching tasks easy though. For example:
MutexGuard</code> can be passed to mem::forget</code> too, but we don't expose a method
for that directly on MutexGuard</code> because doing so causes the lock to be held
forever. Depending on how we feel about dangling tasks, we may want to use API
design to disincentivize people from going into unwanted states.</p>
An async trait which can't be cancelled?</h2>
There has been talk in the Async Foundations WG about potentially
creating a new async trait that would create a future which guarantees it
must be run to completion (e.g. "can't be cancelled"). Our existing
core::future::Future</code> trait would inherit from that trait, and add the
additional guarantee that it can in fact be cancelled.</p>
I've seen two main motivations for going in this direction:</p>

FFI compatibility with C++'s async system, where failing to run a "C++
Future" to completion is undefined behavior.</li>
It would make it easier for non-experts to author async Rust by making
"cancellation" something you don't need to learn about up front.</li>
</ol>
This is not the right post to dig into the first point, but the second point
certainly is interesting for us here.  While I agree this may indeed pose a
hurdle for people writing async today, I don't believe it will going forward
because of the work we're doing to eliminate the need to manually author Poll</code>
state machines already. Work is currently ongoing to for example change the
signature of Stream</code> from fn poll_next</code> to async fn next</code>. Similarly the
working group is working on adding async traits and async closures, all with the
goal to reduce the need to author futures by hand.</p>
As we've observed, handling cancellation is most difficult when authoring
futures by hand. As that is where we need to build the resiliency to halts
(through cancellation and errors alike) that the rest of the futures call graph
relies upon. If we make it so that manually authored futures are only required
to implement the primitives that the rest of the ecosystem relies upon, then the
problem has already been solved.</strong></p>
To close this out: I think Rust's general approach to ensuring {IO
safety</a>, memory
safety, halt safety} should be multi-pronged. We should make it so that for the
vast majority of operations, operators shouldn't need to reach for Rust's
powertools. But for when powertools are definitely the way to go, we should add
all the lints and hints we can to ensure they're operated safely 10</a></sup>.</p>
^10</sup>For example, if I ever need to author something using
[MaybeUninit</code>], I want the compiler to remind me to author a drop
guard</a>
when keeping it live past something which may panic. Perhaps someday (:</p>
</div>
intermediate matching on cancellation?</h2>
While editing this post, I heard of a possible third motivation to potentially
have an alternate future trait: wanting the ability to match on cancellation.
The idea is that we can replace ?</code> with match</code> to handle errors using
alternate logic, but we cannot do at a cancellation point for .await</code>.</p>
// Using the Try operator to re-throw the error.
</span>let</span> string = fs::read_to_string("</span>my-file.txt</span>").await?;
</span>
</span>/// Manually re-throwing the error.
</span>let</span> string = </span>match </span>fs::read_to_string("</span>my-file.txt</span>").await {
</span>    Err(err) => </span>return </span>Err(err),
</span>    Ok(s) => s,
</span>};
</span>
</span>// We can replace the `?` with `match`, but we can't replace `.await`
</span>// with anything else.
</span></code></pre>
The idea is that in conjunction with an un-cancellable future trait,
cancellation would instead be performed by sending a signal to the underlying IO
resource we're waiting on, which would in turn stop what it's doing and return
an io::ErrorKind::Interrupted</code> error that should manually be bubbled up by
intermediate futures to where it can be matched on by the caller.</p>
This argument may sound appealing: right now we indeed can't destructure
.await</code> into a match</code> statement the way we can with ?</code>. So maybe this
mechanism would be useful?</p>
To dive in a little; let's assume we have the following tree of futures:</p>
main
</span>  -> foo (times out after 1 sec)
</span>    -> bar (times out after 3 secs)
</span>      -> task::sleep (wait for 2 secs)
</span></code></pre>
If we want to handle the timeout of foo (times out after 1 sec)</code> in our main
function, we could just match on it in either case:</p>
// Current method of handling async cancellation.
</span>let</span> dur = Duration::from_secs(</span>1</span>);
</span>let</span> string = </span>match </span>fs::read_to_string("</span>my-file.txt</span>").</span>timeout</span>(dur).await {
</span>    Err(err) => panic!("</span>timed out!</span>"),
</span>    Ok(res) => res?,
</span>};
</span>
</span>// Runtime-signal cancellation.
</span>let</span> dur = Duration::from_secs(</span>1</span>);
</span>let</span> string = </span>match </span>fs::read_to_string("</span>my-file.txt</span>").</span>timeout</span>(dur).await {
</span>    Err(ErrorKind::Interrupted) => panic!("</span>timed out!</span>"),
</span>    Err(err) = </span>return</span> err,
</span>    Ok(s) => s,
</span>};
</span></code></pre>
In practice both approaches have incredibly similar semantics at the calling
side. The main difference is that in the runtime-signal approach we may never
hit the error path. In fact, there is no longer a guarantee that the inner
futures propagate the cancellation. They can instead choose to ignore the
cancellation and (erroneously) attempt to retry. Retries should always be
scheduled alongside timeouts; that way if we timeout once we can retry again. If
the two are de-coupled, retries beyond the first will not have an associated
timeout, and risk hanging forever. This is undesireable, and something we should
steer people away from. And our existing semantics already achieve that.</p>
There may be reasons why we might want to detect cancellation on intermediate
futures, for example for the purpose of providing internal
logging</a>. But this
is already possible by combining completion status
tracking</a>
with Drop</code> guards.</p>
This is further complicated by the fact that we're now overloading
io::ErrorKind::Interrupted</code> to carry cancellations from both the operating
system, and trigger in user space. Errors cause by the operating system should
be retried in-place. But errors cause by users should always bubble up. We can
no longer tell them apart.</p>
Another issue is that in order to uniformly support cancellation of futures
futures must now need to carry an io::Result</code> in their return type. We'd need
to rewrite I/O primitives such as
task::sleep</code></a>
to be fallible just for this purpose. It's not the worst; but it is reminescent
of the Futures 0.1 days, where futures had always had to be
fallible</a>.</p>
Overall I do think both approaches are roughly comparable. But because the
cancellation-through-signals variant enables cancellations to be ignored
(accidental, or otherwise), it exposes a set of footguns to users of async that
our current system does not have 11</a></sup>.</p>
^{11</sup>
This is probably worth an entire blog post on its own. But I
have so many posts on in-progress already that I figured I'd add it in here.</p>
</div>}Defer blocks?</h2>
The mechanisms to guard against halt-safety are similar to those for unwind
safety</a>: create a
drop guard to ensure we run our destructors. Manually writing these can get
pretty
verbose</a>.</p>
Languages such as Swift and Go provide a language-level solution to this, in the
form of the defer</code> keyword. This effectively allows users of the language to
write an in-line drop guard to create an anonymous destructor. "The defer
keyword in Swift: try/finally done
right"</a> is a thorough
overview of how this works in Swift. But for illustrative purposes, here's an
example:</p>
func </span>writeLog() {
</span>    </span>let</span> file = openFile()
</span>    </span>defer</span> { closeFile(file) }
</span>
</span>    </span>let</span> hardwareStatus = fetchHardwareStatus()
</span>    </span>guard</span> hardwareStatus != </span>"disaster" </span>else</span> { </span>return </span>/* defer block runs here */</span> }
</span>    file.write(hardwareStatus)
</span>
</span>    </span>// defer block runs here
</span>}
</span></code></pre>
Rust's scopeguard crate</a>
provides a collection of defer</code> macros. But these wouldn't suffice for the
example we shared
earlier</a>,
since we want to maintain access to the data while continuing to access it, and
scopeguard::defer</code> doesn't let
us</a>.
This is because what we want is drop guards to not take ownership of the value
until the destructor is run. I believe this is also referred to as late
binding</em>. And the best way we could achieve that would be by introducing a
language feature.</p>
To be clear: I'm not necessarily advocating we introduce defer</code> into Rust. It
would add a form of non-linear control flow to the language which could confuse
a lot of folks 12</a></sup>. But if we consider {halt, unwind}</code>-safety to be
important enough that we want to provide users with better tools; then defer</code>
seems like a candidate we may want to explore more closely.</p>
^{12</sup>
Folks have told me this from using other languages. I've never
actually used defer</code>, so I can't comment on what it's like. But I have done a
lot of non-linear control flow by writing callback-heavy JavaScript, and that
indeed takes some getting used to.</p>
</div>}update (2021-11-14):</em> as folks correctly pointed out, while
scopeguard::defer!</code> does not provide provide access to the captured values
before dropping,
scopeguard::ScopeGuard</code></a>
does via the Deref</code> / DerefMut</code> traits. The flexibility of being able to
remove the Drop</code> impl of the guard by converting the guard into its inner value
makes for a compelling argument that solving in-line drop impls would be better
solved through a library addition than through a language item.</p>
Cancellation and io_uring</h2>
This section was added on 2021-11-17, after the post was initially published.</em></p>
Patrick Walton</a> asked the following question
/r/rust</a>:</p>

One motivation for completion futures that either wasn't mentioned or is an
unexpected side effect of point #1 (compatibility with C++) is that async code
in C/C++ can take non-owning references into buffers. For example, on Linux if
you issue an async read() call using io_uring and you later get cancelled, you
have to tell the kernel somehow that it needs to not touch the buffer after Rust
frees it. There are ways to do this, such as giving the kernel ownership of the
buffers using IORING_REGISTER_BUFFERS, but having the kernel own the I/O buffers
can make things awkward. (Async C++ folks have shown me patterns that would
require copies in this case.) Have you all given any thought as to the best
practices here? It's a tough decision, as the only real solution I can think of
involves making poll() unsafe, which is unsavory (NB: not necessarily wrong).</p>
</blockquote>
Saoirse</a> wrote an excellent
overview</a> of how the completion-based
Linux io_uring</code> family of kernel APIs</a>
interacts with cancellation in Rust. The whole post is worth a read as it covers
much of the nuance and safety considerations involved when using io_uring</code> with
Rust. But it also directly answers Patrick's question:</p>

So I think this is the solution we should all adopt and move forward with:
io-uring controls the buffers, the fastest interfaces on io-uring are the
buffered interfaces, the unbuffered interfaces make an extra copy. We can stop
being mired in trying to force the language to do something impossible.</p>
</blockquote>
Conclusion</h2>
In this post we've looked at how cancellation works for futures and tasks. We've
covered how cancellation propagation ought to work, and how we can backport it
to existing runtimes. And finally we've covered how to reason about structured
concurrency, how cancellation propagation can be applied to groups of tasks, and how
async cancellation might interact with async designs currently being drafted.</p>
I hope this was informative! You might have noticed the title of this post has a
"1" appended to it.  I'm planning to post a follow-up to this post covering the
design space of triggering cancellation at a distance, and possibly another on
task grouping.  working on different posts on async concurrency, so expect more
on that soon.</p>
I've gone and opened a discussion thread on
Internals</a>.
And if you enjoyed this post, and would like to see whatever I'm dreaming up in
a real-time fashion, you can follow me
@yoshuawuyts</a>.</p>
Update (2021-11-14):</em> Firstyear wrote a follow-up to this
post</a>
on transactional operations in Rust, and how they interact with (async)
cancellation and halt-safety. If you're interested in transactions and
rollbacks, I recommend giving it a read!</p>
Thanks to: Eric Holk, Ryan Levick, Irina Shestak, and Francesco Cogno for
helping review this post prior to publishing. And thanks to Niko Matsakis for
walking me through some of the alternate futures (lol) of the future trait.</em></p>

Futures Concurrency II: A Trait Approach
Thu, 02 Sep 2021 00:00:00 +0000
It's been exactly two years since I wrote about futures
concurrency</a>. Some work has
happened since, and I figured it would be a good time to recap futures
concurrency, and share some of the new developments.</p>
In this post we'll establish a base model of async concurrency for Rust. We'll
cover ergonomic new APIs to implement said model. And finally we'll describe a
plausible path towards inclusion of those APIs in the stdlib in the short-term.</p>
Modes of concurrency</h2>
Concurrency can be thought of as awaiting multiple things at the same time on
the same thread. Generally we can divide the various modes of concurrency up
along two axes:</p>

Do we want to wait for all outputs, or just the first output?</li>
When an error occurs, do we keep going or do we return early ("short circuit")?</li>
</ol>
As of August 2020, JavaScript has had the following Promise</code> concurrency APIs
available on most platforms:</p>
</th> Wait for all outputs</strong></th> Wait for first output</strong></th></tr></thead>

Continue on error</strong></td> Promise.allSettled</code></a></td> Promise.any</code></a></td></tr>
Return early on error</strong></td> Promise.all</code></a></td> Promise.race</code></a></td></tr>
</tbody></table>
JavaScript covers most concurrency uses really well. Rust has a counterpart to
JavaScript's Promise</code> type in Future</code>, but it differs in two key ways:</p>

Promise</code>s are always fallible, while Future</code>s have the option not to be.</li>
Promise</code>s start executing immediately, while Future</code>s need to be invoked
first. 1</a></sup></li>
</ol>
^1</sup>In terms of execution, the rough equivalent of a Promise</code> in
JavaScript is
Task</code></a> in Rust
(starts executing immediately on creation).
The rough equivalent of a Future</code> in Rust is a
"thenable"</a>
in JavaScript (starts executing once await</code>ed).</p>
</div>
For async-std</code></a> we've been
experimenting with introducing similar concurrency APIs as those found for
JavaScript Promise</code>s as part of our future</code> submodule. For Rust Futures</code>
which don't return Result</code> we expose two methods for concurrency:</p>
Wait for all outputs</strong></th> Wait for first output</strong></th></tr></thead>

Future::join</code></td> Future::race</code></td></tr>
</tbody></table>
When we add fallability</em> to the mix (e.g. return a Result</code> from the future)
async-std</code>'s try_</code> methods for concurrency become available we can compare it
1:1 to JavaScript's APIs:</p>
</th> Wait for all outputs</strong></th> Wait for first output</strong></th></tr></thead>

Continue on error</strong></td> Future::join</code></td> Future::try_race</code></td></tr>
Return early on error</strong></td> Future::try_join</code></td> Future::race</code></td></tr>
</tbody></table>
The try</code> prefix can be thought of as inverting the semantics of the method. In
the case of try_join</code> we no longer wait for all</em> items to complete; on error
we'll now exit early and cancel all others. In the case of try_race</code> we no
longer wait for the first</em> item to complete; on error we'll keep waiting until
we either find a successful item or run out of items.</p>
This is the model we wrote about two years ago, and aside from a rename
(select</code> -> race</code>) it's remained identical. However despite that, we still
haven't attempted to introduce this functionality into the stdlib yet. And
that's because of missing ✨ language features ✨.</p>
Joining tuples</h2>
In my post "future::join</code> and
const-eval"</a> I talk
about the issue of joining multiple futures:</p>

[...] once we start joining more than two futures, the output types become different:</p>
</blockquote>
let</span> a = future::ready(</span>1</span>u8</span>);
</span>let</span> b = future::ready(</span>2</span>u8</span>);
</span>let</span> c = future::ready(</span>3</span>u8</span>);
</span>assert_eq!(join!(a, b, c).await, (</span>1</span>, </span>2</span>, </span>3</span>));
</span>
</span>let</span> a = future::ready(</span>1</span>u8</span>);
</span>let</span> b = future::ready(</span>2</span>u8</span>);
</span>let</span> c = future::ready(</span>3</span>u8</span>);
</span>assert_eq!(a.</span>join</span>(b).</span>join</span>(c).await, (</span>1</span>, (</span>2</span>, </span>3</span>))); </span>// oh no!
</span></code></pre>

As you can see, each invocation of Future::join returns a tuple. But that
means that chaining calls to it starts to nest tuples, which becomes hard to
use. And it becomes more nested the more times you chain. Oh no!</p>
</blockquote>
In the post we explain that until then we'd been solving this 2</a></sup> by using a join!</code>
macro, which by nature is variadic 3</a></sup>. However ideally we would be
able to write methods which are generic over tuples. Not only would that allow
us to write a better join</code> function, it would also save us from having stdlib
twelve different PartialEq</code> and PartialOrd</code>
impls</a>:</p>
^{2</sup>
"this" means having nested return types like Join<Join<Join<A>, B>, C></code> which return tuples in the shape of (a, (b, c))</code>, etc. What we want is to
be able to have a single "flat" Join<A, B, C...></code> return type which returns
(a, b, c)</code> without any nesting. That's the challenge we're trying to solve.</p>
</div>}^{3</sup>
"Variadic" means "can support any number of arguments". For example
println!</code> can  print out any number of arguments. We can't express that using
regular Rust functions yet.</p>
</div>}Just like const
generics</a> solved the
a similar issue for arrays, it seems likely like we'll eventually want to solve
this for tuples too. But that's unlikely to happen anytime soon (if I'd hazard a
guess, I'd say if work starts on this in 2022, then maybe we could have it by
2023?). So for now we need to work around this restriction, and luckily there
are a few options.</p>
How to expose futures concurrency</h2>
In response to "future::join</code> and
const-eval"</a>, matthieum
pointed
out</a>
we can invert the API instead. Rather than exposing this functionality as functions scoped under
std::future</code>, we could instead extend in-line container types with traits that
allow for more fluent composition. I wrote the
futures-concurrency</code></a>
crate to test out that design, and the design feels incredibly</em> good. For
example, this is what it looks like to await multiple similarly-typed futures
without any intermediate allocations:</p>
use </span>std::future;
</span>use </span>futures_concurrency::prelude::*;
</span>
</span>let</span> a = future::ready(</span>1</span>u8</span>);
</span>let</span> b = future::ready(</span>2</span>u8</span>);
</span>let</span> c = future::ready(</span>3</span>u8</span>);
</span>assert_eq!([a, b, c].</span>join</span>().await, [</span>1</span>, </span>2</span>, </span>3</span>]);
</span></code></pre>
And this is for multiple differently-types futures without any intermediate
allocations:</p>
use </span>std::future;
</span>use </span>futures_concurrency::prelude::*;
</span>
</span>let</span> a = future::ready(</span>1</span>u8</span>);
</span>let</span> b = future::ready("</span>hello</span>");
</span>let</span> c = future::ready(</span>3</span>u16</span>);
</span>assert_eq!((a, b, c).</span>join</span>().await, (</span>1</span>, "</span>hello</span>", </span>3</span>));
</span></code></pre>
But sometimes we want to allocate because we don't know how many futures we're
going to want to await in parallel. So it works with Vec<Future></code> as well,
similar to futures</code>' join_all</code> function:</p>
use </span>std::future;
</span>use </span>futures_concurrency::prelude::*;
</span>
</span>let</span> a = future::ready(</span>1</span>u8</span>);
</span>let</span> b = future::ready(</span>2</span>u8</span>);
</span>let</span> c = future::ready(</span>3</span>u8</span>);
</span>assert_eq!(vec![a, b, c].</span>join</span>().await, vec![</span>1</span>, </span>2</span>, </span>3</span>]);
</span></code></pre>
This feels like the most promising approach so far. Once the right traits are in
scope (2024 edition, anyone?), the way to await becomes intuitive. We no longer
need to chain calls to .join().join().join()</code> or wrap items in awkward macros.
Instead we group futures together, suffix it with our preferred method of
concurrency, and then types pop out on the other end. Easy.</p>
There are some caveats to this though. The way we've implemented this for tuples
is the same way the stdlib implements PartialOrd</code> and PartialEq</code>: by using a
macro. However because we're returning custom types from these traits, it means
we're currently generating twelve different Join</code>
types</a>.
Surely there must be a way around this right?</p>
Async Traits and TAITs</h2>
Currently the futures_concurrency::Join</code> trait is defined like this:</p>
trait </span>Join {
</span>    </span>type </span>Output;
</span>    </span>type </span>Future: Future<Output = </span>Self::</span>Output>;
</span>    </span>fn </span>join</span>(</span>self</span>) -> </span>Self::</span>Future;
</span>}
</span></code></pre>
We have two types: the Output</code> type which is what our method will return once
awaited. And the Future</code> type which is the future our join</code> method will return
that needs to be awaited. However the async foundations WG is working on
introducing "async traits" into the language. Once that's in place it's expected
we'll be able to rewrite the trait like so:</p>
trait </span>Join {
</span>    </span>type </span>Output;
</span>    async </span>fn </span>join</span>(</span>self</span>) -> </span>Self::</span>Output;
</span>}
</span></code></pre>
It's expected that all async traits will go this way, since it removes the need
to manually implement futures using
Pin</code></a> 4</a></sup> (which is a big
improvement for ergonomics). However this comes with an issue: How do we
manually return futures?</p>
^{4</sup>
I consider Pin</code> to be like unsafe {}</code> in that it's essential for Rust
to expose this - but under no circumstance should it be required to perform
common operations.</p>
</div>}The answer for that appears to be
TAITS</a> ("Type Alias Impl
Trait"). This allows traits to be used in type aliases, and treated as concrete
types. Async traits hasn't been RFC'd yet, but I expect we'll want to enable it
to work with TAITS once they stabilize. So that would allow us to implement the
async trait differently depending on whether we want to name the output type or
not:</p>
// Implement the trait using `async fn`.
</span>impl</span><T> Join </span>for </span>Vec<T> {
</span>    </span>type </span>Output = Vec<T>;
</span>    async </span>fn </span>join</span>(</span>self</span>) -> </span>Self::</span>Output;
</span>}
</span>
</span>// Implement the trait using with a named future.
</span>type </span>JoinFuture<T> = </span>impl </span>Future<Output = Vec<T>>;
</span>impl<T> Join for Vec<T> {
</span>    </span>type </span>Output = Vec<T>;
</span>    </span>fn </span>join</span>(</span>self</span>) -> JoinFuture<T>;
</span>}
</span></code></pre>
Perhaps you're seeing where I'm going with this, but for tuples we can then
create a shared Join</code> future type for all tuple variants without needing to
have variadics in the language:</p>
type </span>JoinFuture<T> = </span>impl </span>Future<T>;
</span>
</span>// Implement `Join` for tuple `(A, B)`.
</span>impl<T, </span>const</span> N: </span>usize</span>> Join for (A, B)
</span>where
</span>    A: Future,
</span>    B: Future,
</span>{
</span>    </span>type </span>Output = (A::Output, B::Output);
</span>    </span>fn </span>join</span>(</span>self</span>) -> JoinFuture<</span>Self::</span>Output> { ... }
</span>}
</span>
</span>// ...and repeat for all other tuples (likely using a macro).
</span>impl</span><T, </span>const</span> N: </span>usize</span>> JoinTrait </span>for</span> (A, B, C) { ... }
</span>impl</span><T, </span>const</span> N: </span>usize</span>> JoinTrait </span>for</span> (A, B, C, D) { ... }
</span>impl</span><T, </span>const</span> N: </span>usize</span>> JoinTrait </span>for</span> (A, B, C, D, E) { ... }
</span></code></pre>
And once we have variadics as part of the language, we could migrate the
implementations to use that instead:</p>
type </span>JoinFuture<T> = </span>impl </span>Future<T>;
</span>
</span>// Implement `Join` for all tuples of two or more items.
</span>impl<A, B, </span>const</span> N: </span>usize</span>> Join for (A, B)
</span>where
</span>    A: Future,
</span>    B...: Future, </span>// <- Entirely made up variadic syntax
</span>{
</span>    </span>type </span>Output = (A::Output, B::Output);
</span>    </span>fn </span>join</span>(</span>self</span>) -> Join<</span>Self::</span>Output>;
</span>}
</span></code></pre>
It's important to emphasize though that the semantics of TAITS have not yet been
stabilized, and are subject to change. But the introduction of TAITS may also
have implications for the rest of the language; for example bringing inference
to manual trait implementations too. 5</a></sup></p>
^{5</sup>
An example of this could be impl Iterator for Foo { fn next(&mut self) -> Option<i32> { ... } } }</code>.
The associated type Item</code> can directly be inferred from fn next</code>'s return
type. If this was allowed, it would bring the semantics of manual trait
implementations closer to those done via TAITs.</p>
</div>}Road to std</h2>
Despite what you might think after the last section, we don't need to wait until
we have async traits, TAITS, or variadics in the stdlib to provide a good
experience for async concurrency. Instead we can start small, and as more
language features stabilize, incrementally expose more functionality.</p>
The most basic form in which we could introduce Join</code> into the stdlib would be
by making it available for implementation within the stdlib-only using the
C-SEALED future-proofing
pattern</a>:</p>
trait </span>Join: private::Sealed {
</span>    </span>type </span>Output;
</span>    </span>type </span>Future: Future<Output = </span>Self::</span>Output>;
</span>    </span>fn </span>join</span>(</span>self</span>) -> </span>Self::</span>Future;
</span>}
</span>
</span>mod </span>private {
</span>    </span>pub</span>(</span>super</span>) </span>trait </span>Sealed {}
</span>
</span>    </span>pub</span>(</span>super</span>) </span>struct </span>Join2 { ... }
</span>    </span>impl </span>Future </span>for </span>Join2 { ... }
</span>    </span>impl </span>super::Join </span>for</span> (A, B) { </span>type </span>Future = Join2<A, B>; ... }
</span>
</span>    </span>// ...etc
</span>}
</span></code></pre>
Because in addition to having the trait be sealed, the returned futures are
private too. The only way to reference the associated futures is through impl Future</code>. Which lines up perfectly with async traits, which would allow us to
drop the Seal</code> and publicly expose the trait as core::future::Join</code> like this:</p>
trait </span>Join {
</span>    </span>type </span>Output;
</span>    async </span>fn </span>join</span>(</span>self</span>) -> </span>Self::</span>Output;
</span>}
</span></code></pre>
This version still only allows storing the future as impl Future</code>, but once
TAITs land can be extended to named futures instead. And then eventually with
variadics make the final jump to feature completedness.</p>
In summary: if we wanted to, we could start implementing a fully forward
compatible version of Join</code> and related APIs in the stdlib today</em>!</p>
Conclusion</h2>
In this post we've established a basic model for concurrently executing multiple
futures. We've described an ergonomic new API, prototyped in the
futures_concurrency</code> crate which enables concurrent execution of futures in a
more ergonomic way than was previously possible. And finally we've shown a path
for gradually introducing these APIs into the stdlib in a forward-compatible
manner.</p>
For me, the most important point of this post is to show that it's possible to
create Futures concurrency APIs which feel natural to use, and can be called
with little effort. The exact shape of the traits is less important, and
something we can improve on over time. We've shown that we can introduce these
methods in the stdlib in a forward-compatible manner, giving us time to polish
and iterate on these traits as more language features become available.</p>
One mode of execution we didn't cover in this post is a variation on race</code> in
which multiple futures are resolved concurrently, and values are yielded as soon
as they're resolved. This is what the futures_rs::select!</code> macro is commonly
used for. In a future installment of this series we'll take a look at how we
could potentially bring that functionality to the stdlib as well.</p>
Another thing which we haven't covered yet is how to introduce the try</code>
variants of the APIs. The obvious choice would be to introduce a TryJoin</code> trait,
but that may not quite be the right fit for what we're trying to do. In a future
installment we'll look more closely at introducing the try</code> variants.</p>
My plan for the futures-concurrency</code> crate is to keep working on it, and bring
it to a state where all the APIs shared in this post have been implemented. Once
that's proven to work, and we're sure that this method is forward compatible, we
can consider proposing this for addition to the stdlib.</p>
But that's something for the future. For now I hope you enjoy the
futures-concurrency</code> crate, and I'd be keen to hear what folks think!</p>
Special thanks to: Sy Brand</a> for help with
understanding variadics in C++, Scott McMurray</a>
for elaborating on TAITs, and Ryan Levick</a>
and Eric Holk</a> for reviewing this post
prior to publishing.</em></p>

Async overloading
Tue, 24 Aug 2021 00:00:00 +0000
If the stdlib were to expose APIs for async IO, how should it be namespaced?</p>
// Like this?
</span>use </span>std::async_io::remove_file;
</span>
</span>// Like this?
</span>use </span>std::io::async_remove_file;
</span>
</span>// Or like this?
</span>use </span>std::io::remove_file;
</span></code></pre>
This post looks at async overloading in Swift, how we could potentially
translate it to Rust, and makes the case that we should make investigating its
feasibility a priority for the Async Foundations WG.</p>
Choose your calling convention</h2>
When designing an API in Rust which performs IO, you have to make a decision
whether you want it to be synchronous, asynchronous, or both.</p>
If you choose "synchronous" it's likely easier to write, iterate on,
and use. But it may be less performant, and interacting with async APIs can be
tricky.</p>
Choosing "asynchronous" will often mean better performance (especially on
multi-core machines), better compatibility with a lot of other networking
operations. But it is often significantly harder to author and maintain.</p>
If you author both so you can let end-users decide what they want to
consume, not only will you incur the cost of writing the same code twice. You
also need to choose how to expose both APIs to end-users. This leads to
situations like with the mongodb</code>
crate</a> which exposes a
feature-flagged sync</code></a>
submodule which exposes a copy of all IO types for use in synchronous code.</p>
There's a clear "primary API" (async), and "secondary API" (sync). From a
maintainer's perspective this means that whenever a new async API is introduced,
you need to manually ensure that the sync counterpart is used as well. And from
a user's perspective you need to make sure you don't accidentally call the wrong
API since both live in the same namespace.</p>
Another example are the postgres</a>
and tokio-postgres</code></a>
crates which split the sync/async APIs out into two different crates instead.
This solves the namespace problem, but still requires manually ensuring both
APIs are updated.  And end-users still need to remember to import the right
crate for their purpose.</p>
The current state of crates isn't unworkable</em>: we know Rust is used in
production, often to great success. But I don't think it's hard to argue that it
isn't particularly great</em> either. The fact that we need feature flags,
submodules, or even separate crates is a direct artifact of the limitations of
Rust as a language required to create good designs</em>, and not an issue with any
of the ecosystem crates.</p>
The standard library</h2>
There's no arguing that Async Rust is hot right now. Companies both large and
small have made commitments to using Rust, and Async in particular is integral
to Rust's adoption. The Async WG (which I'm part of) is looking at integrating
parts of the ecosystem into the stdlib; including core traits, but also
networking abstractions other IO facilities. And unfortunately the logical
endpoint of our current trajectory would be a significant increase in API
surface of the stdlib</strong>.</p>
As with all things Rust: nothing is certain until it's stabilized. But "async
compat" has been a recurring theme, and the Async WG is looking at ways to
include runtime-independent async APIs in the
stdlib</a>.
Which if we were to add an async counterpart for every IO API currently in the
stdlib, we'd end up exposing over 300 new
types</a> — not accounting for methods.</p>
To get a sense of what this would feel like we can look at other language
runtimes.
Python</a>,
C#</a>,
and Node.js all have this sync/async split. I'm most familiar with Node.js which
has 3 different calling conventions (callbacks, async/await, blocking). Which in
practice looks like this:</p>
// Delete a file with the default async callback-based API.
</span>import </span>{ </span>unlink </span>} </span>from </span>'</span>fs</span>';
</span>unlink</span>('</span>./tmp/foo.txt</span>', (</span>err</span>) </span>=> </span>{
</span>  </span>if </span>(</span>err</span>) </span>throw </span>err</span>;
</span>  </span>console</span>.</span>log</span>('</span>success</span>');
</span>});
</span></code></pre>
// Delete a file with the recommended async Promise-based API.
</span>import </span>{ </span>unlink </span>} </span>from </span>'</span>fs/promises</span>';
</span>await </span>unlink</span>('</span>./foo.txt</span>');
</span>console</span>.</span>log</span>('</span>success</span>');
</span></code></pre>
// Delete a file with the sync blocking API.
</span>import </span>{ </span>unlinkSync </span>} </span>from </span>'</span>fs</span>';
</span>unlinkSync</span>('</span>./foo.txt</span>');
</span>console</span>.</span>log</span>('</span>success</span>');
</span></code></pre>
Rust's counterpart to Node's fs.unlink</code> is
std::fs::remove_file</code></a>. Say we wanted to expose an async counterpart to it as part of the stdlib. We'd have a few options:</p>

We could create a new submodule (e.g. std::async_fs</code>) to contain a full
counterpart of std::fs</code>, including std::async_fs::remove_file</code>. This approach
is similar to how Python supports both
import io</code></a> and
import asyncio</code></a>.</p>
</li>

We could add async types directly under std::fs</code>. So we'd
have both std::fs::remove_file</code> and std::fs::async_remove_file</code>. This approach
is similar to the way Node.js's exposes sync APIs. But having to prefix
each call with async_*</code> and suffix with .await</code> doesn't make this an appealing
option for Rust.  Which is likely why I haven't heard anyone who prefers this
approach over submodules. Even if prefixing submodules could lead to gems such
as std::async_sync</code></a>
1</a></sup>.</p>
</li>
</ol>
^{1</sup>
I don't believe the libs team would seriously let async_sync</code> be a
part of the stdlib. I've heard talk about deprecating std::sync</code> and splitting
it into separate submodules such as std::lock</code> and std::atomic</code>. Which means
we'd likely sooner see a std::async_lock</code> than std::async_sync</code>. But I still
find it amusing to think about an alternate future where we'd have sync</code>,
async_sync</code>, async</code>, Async</code>, Sink</code>, AsyncSink</code> and Sync</code> in the stdlib.</p>
</div>
But both options share a core issue: APIs still need to be manually matched to
the context they're called in. If you're in a sync context,
std::async_fs::remove_file</code> cannot be called. You need to create an async
context first.</p>
And likewise if you're in an async context: std::fs::remove_file</code> can</em> be
used, but it's most likely not what you want: you probably want the async
version instead. This means that depending on the type of context you're in,
half the IO APIs in the stdlib would be of no use to you. Which leaves the
question: Is there a way we could prevent manually needing to match APIs in
the stdlib to their right calling convention?</strong></p>}Async overloading in Swift</h2>
In an amendment to
SE-0296: Async Await</a> made last month,
Swift has added support for async overloading which solves this exact problem!
It enables sync and async functions of the same name to be declared in the same
scope, leaving it up to the caller's context to determine which variant is the
right one:</p>
// Existing synchronous API
</span>func </span>doSomethingElse() { ... }
</span>
</span>// New and enhanced asynchronous API
</span>func </span>doSomethingElse() async { ... }
</span></code></pre>
Calling async functions from sync functions triggers an error like this:</p>
func </span>f() {
</span>  </span>// Compiler error: `await` is only allowed inside `async` functions
</span>  await doSomething()
</span>}
</span></code></pre>
But if you're trying to call the sync variant of an overloaded function inside
an async function, you'd see the following error:</p>
func </span>f() async {
</span>  </span>// Compiler error: Expression is 'async' but is not marked with 'await'
</span>  doSomething()
</span>}
</span></code></pre>
However it's still possible to call the synchronous version of the API by
creating a synchronous context inside the async block:</p>
func </span>f() async {
</span>  </span>let</span> f2 = {
</span>    </span>// In a synchronous context, the non-async overload is preferred:
</span>    doSomething()
</span>  }
</span>  f2()
</span>}
</span></code></pre>
But async overloading is not required. It's still possible to declare async
functions without overloading: they'll just only work in async contexts. And
similarly it's possible to declare sync functions which will work in all
contexts.</p>
Unlike other languages, Swift will not need to effectively double the API
surface of their stdlib to support their newly added async functionality. From a
user's perspective the same API will "work as expected" in both sync and async
contexts. And that is something Rust can learn from.</p>
Bringing async overloading to Rust</h2>
Now that you know how Swift has solved this issue, it asks the question: Could
we integrate a system like this into Rust. The fact that I'm talking about this
at all probably gives away that I think we can. But what would that look like?</p>
For async functions, overloading could be easy enough. Adapting the Swift
example we saw earlier, the following should be able to work:</p>
// Existing synchronous API
</span>fn </span>do_something_else</span>() { ... }
</span>
</span>// New and enhanced asynchronous API
</span>async </span>fn </span>do_something_else</span>() { ... }
</span></code></pre>
We could even give a specialization
spin</a>
to the API, and make the syntax something like:</p>
default </span>fn </span>do_something_else</span>() { ... }
</span>async </span>fn </span>do_something_else</span>() { ... }
</span></code></pre>
But I don't know what the right syntax for this would be. Maybe it would warrant
a different approach altogether! Today, if we try awaiting an async function
inside a synchronous closure, we're met with the following diagnostic:</p>
error[E0728]: `await` is only allowed inside `async` functions and blocks
</span> --> src/lib.rs:4:5
</span>  |
</span>3 | fn f() {
</span>  |    - this is not `async`
</span>4 |     do_something().await;
</span>  |     ^^^^^^^^^^^^^^^^^^^^ only allowed inside `async` functions and blocks
</span></code></pre>
With the overload added, we can start suggesting fixes for errors like this too 2</a></sup>.
For example:</p>
^2</sup>As an aside; I'm super excited for the upcoming
diagnostics changes</a>. The diff
view is looking really good! It's just a bummer my blog theme doesn't allow me
to custom-color diagnostics output yet. For now imagine the diagnostics in my
blog have pretty colors like the compiler does, heh.</p>
</div>
  |
</span>help: try removing `.await` from the function call
</span>  |
</span>3 | async fn f() {
</span>4 -     do_something().await;
</span>4 +     do_something();
</span>  |
</span></code></pre>
And calling the overloaded sync function from an async context could produce a
diagnostic like this:</p>
warning: unused implementer of `Future` that must be used
</span> --> src/lib.rs:4:5
</span>  |
</span>3 | async fn f() {
</span>4 |     do_something();
</span>  |     ^^^^^^^^^^^^^^^
</span>  |
</span>  = note: this function is called in an async context and uses the async overload
</span>  = note: `#[warn(unused_must_use)]` on by default
</span>  = note: futures do nothing unless you `.await` or poll them
</span>  |
</span>help: try adding `.await` to the function call
</span>  |
</span>3 | async fn f() {
</span>4 |     do_something().await;
</span>  |                   ++++++
</span></code></pre>
Just like in Swift, using the synchronous overload in an async context could be
done by writing:</p>
async </span>fn </span>f</span>() {
</span>    </span>let </span>f2 </span>= || {
</span>        doSomething()
</span>    };
</span>    </span>f2</span>()
</span>}
</span></code></pre>
For traits we could imagine something similar as for functions. Supposing the
default</code> keyword (specialization) could also be used for overloads; we could
imagine we could write something like this for an async version of
Read</code></a>:</p>
/// The synchronous `Read` trait
</span>pub</span> default </span>trait </span>Read {
</span>    </span>fn </span>read</span>(&</span>mut </span>self</span>, </span>buf</span>: &</span>mut</span> [</span>u8</span>]) -> Result<</span>usize</span>>;
</span>}
</span>
</span>/// The asynchronous `Read` trait
</span>pub</span> async </span>trait </span>Read {
</span>    async </span>fn </span>read</span>(&</span>mut </span>self</span>, </span>buf</span>: &</span>mut</span> [</span>u8</span>]) -> Result<</span>usize</span>>;
</span>}
</span></code></pre>
Depending on how the traits are implemented, users could end up implementing
none, either, or both. Which variants of the trait are implemented could then be
highlighted through the docs, for example:</p>
---------------------
</span>Trait Implementations
</span>---------------------
</span>Debug
</span>FromRawFd
</span>Read
</span>Read (async)
</span>Write
</span>Write (async)
</span></code></pre>
This really is all a rough sketch. I don't know whether using default</code> as a
keyword would make sense here at all, or what the right way of highlighting this
in the docs are. What I hope to achieve with this is to share a glimpse of how
this could work end-to-end for Rust users. Docs, diagnostics, writing, and
consuming code are all different ways to experience the same feature. And I hope
this provides an idea of what an integration could possibly look like for both
library consumers and authors alike.</p>
Potential caveats</h2>
Backwards incompatibility</h4>
As far as I can tell Rust doesn't have anything yet that would preclude us from
going down this path. Indeed adding this would cause a delay in "shipping" async
Rust. But it would make it so we don't need to effectively double the surface
area of the stdlib, which to me is worth it (this is coming from someone who's
helped author an async copy of virtually every stdlib API).</p>
If we decide this is worth exploring, the one thing we shouldn't do is stabilize
std::stream</code> before our explorations are over. I don't believe any other APIs
are included yet in nightly which could pose an issue for async overloading, so
as long as we don't stabilize that while we figure this out, we should be good!
3</a></sup>.</p>
^{3</sup>
And again, this is coming from someone who helped author the Stream</code>
RFC, and added Stream</code> to nightly. Please believe me that if someone wants to
stabilize async iterators, it's me. But I think async overloading is the kind of
change we should consider and definitively rule out before proceeding we attempt
to stabilize the Stream</code> API as-is.</p>
</div>}Overloading existing stdlib functions</h4>
One issue to be aware of is that unlike Swift we cannot immediately fail if a
synchronous overload is selected in an async function. Rust's async models
allows for delayed .await</code>ing, which means we cannot error at the call-site.
Instead we'll likely need to hook into the machinery that enables #[must_use]</code>;
allowing us to validate whether the returned future is actually awaited — and
warn if it's not. Even though this is slightly different from Swift appears to
do things, it should not present an insurmountable hurdle.</p>
Feasibility of async parity</h4>
One of the goals of the async-std</a>
project was to gauge the feasibility of creating a dual of the stdlib's IO APIs
using async Rust. And two years in we can conclude it's been an overwhelming
success: the only stdlib APIs we haven't been able to faithfully reimplement in
async Rust are those which require async closures and async traits
4</a></sup>. But now that the Async Foundations WG is adding those
capabilities to the language, there are no blockers remaining for API parity
between sync and async Rust.</p>
^4</sup>An example of an API which requires async traits is
async-std</code>'s
Stream::collect</code></a>.
We've managed to work around it, but had to box some items. An example of an API
which requires async closures is
Stream::filter</code></a>.
The callback provided to the method is not async right now because there's no
way to express the lifetime of the borrowed item. But once we have async
closures this will become possible to express. Pretty much all differences
between std and async-std are minor issues like these, which we're fairly
certain can be solved once Async Rust gains more capabilities.</p>
</div>
Past async overloading: code reuse</h4>
Finally, there's an interesting question to ask whether it's possible to reuse
code past an async overload. You could imagine a sync and an async parser
containing identical logic, except for the internal use of .await</code>. To which
degree does Swift's model limit code reuse? That should be something the async
WG should answer once it decides async overloading is indeed desireable, and
starts looking at ways it could be implemented.</p>
Returning impl Future</code> instead of async fn</code></h4>
Without
TAIT</a>s async fn</code>s cannot be named, and thus for the stdlib it'll be desireable to return
manual futures from APIs instead. This means that we might want to write
something like this:</p>
default </span>fn </span>do_something_else</span>() { ... }
</span>fn </span>do_something_else</span>() -> DoSomethingElse { ... }
</span>
</span>struct </span>DoSomethingElse;
</span>impl </span>Future </span>for </span>DoSomethingElse { ... }
</span></code></pre>
Luckily the compiler is aware of which traits are implemented on types, and
could validate that this indeed resolves to a valid async overload.
We should probably even make it so -> impl Future<Output = io::Result<()>></code>
works, but -> io::Result<impl Future<Output = ()>></code> does not. This would likely
be quicker to validate, and keep things feeling like "async fn overloading" only.</p>
Overloads for other effects</h4>
A question which has come up is whether this mechanism could be used for other
effects as well. For example, it could be conceivable that in a try fn</code> we
could expect to call a fallible API, while in a non-try fn</code> we'd expect the
infallible method:</p>
pub</span> default </span>fn </span>reserve</span>(&</span>mut </span>self</span>, </span>additional</span>: </span>usize</span>);
</span>pub</span> try </span>fn </span>reserve</span>(&</span>mut </span>self</span>, </span>additional</span>: </span>usize</span>) -> throws ReserveError;
</span></code></pre>
In my Fallible Iterator Adapter</a> post I pointed
out that we're missing a fair number of fallible variants of existing APIs.
Boats has since authored The Problem of Effects in
Rust</a> which covers the
problem from a language perspective, and shows how if we introduced a different
kind of closure we could eliminate the need for try_</code> iterator adapters.</p>
RFC 2116: Fallible Collection
Allocation</a> covers adding try_</code>
variants of the base building blocks. But the implementation has decided not to add
try_</code> variants for all allocation APIs since that would increase the API
surface too much. Which is a similar reasoning given for why we don't have
fallible variants for all iterator APIs either.</p>
The issues we're covering in this post, Boats' effect post, and the issues in
RFC 2116 all seem related to each other. I don't know if "fallible overloading"
could provide a partial way out of it, but maybe there's something to it? The
possibilities different kinds of overloading could bring certainly are
interesting, and probably worth thinking about more.</p>
Other</h4>
There are two topics which I haven't thought about, but should be thought about
by the working group:</p>

How does async overloading interact with dyn Trait</code>?</li>
How does async overloading interact with FFI?</li>
</ol>
Basing on what I've seen come up in conversations there might be ways to make
this work, but it should be looked at more closely as part of a feasibility
probe.</p>
Conclusion</h2>
Currently the async foundations roadmap does include a note on "async
overloading"</a>,
but it's marked as "slightly past the horizon". I think given this has
implications for async iteration, async IO, and every other async libs API, we
should make sure we've properly investigated async overloading before we make any
changes to the stdlib which would be incompatible with it.</strong></p>
To summarize what we've covered so far:</p>

Async Rust is currently on a trajectory to doubling the API surface of IO
APIs in the stdlib.</li>
Async overloading could provide a way out for us, enabling us to add async IO
APIs to the stdlib while keeping the number of IO APIs in the stdlib to its
current number.</li>
As long as we don't stabilize Stream</code>, async IO traits, or any other async IO
types we should be capable of introducing async overloading without any
sharp edges.</li>
Because async overloading informs the implementation of all of the async
types in the stdlib, investigating its feasiblity should be a priority for the
async WG.</li>
</ol>
My hopes of this post is that it convinces folks that async overloading is an
idea worth taking seriously before we continue with any other designs. And to
perform a feasibility study whether this could actually work from a language,
libs, and compiler perspective.</p>
Thanks to: Eric Holk, Irina Shestak, James Halliday, Scott McMurray, and Wesley
Wiser for helping review this post prior to publishing.</em></p>

optimizing hashmaps even more
Sat, 08 May 2021 00:00:00 +0000
In our last post we took a look at possible ways we could improve the ergonomics
of Rust's refcounting APIs. In this post we'll be looking at Hashmap: how it's
currently implemented, how we could optimize it further, and finally directions
we could explore to enable the compiler to automatically apply these optimizations.</p>
Disclaimer:</strong> I'm not an expert. I'm not on any team involved with const
execution, nor do I hold any decision making power. This exists to share ideas
and curiosities of what might be possible in Rust in the spirit of sharing
posts with pals.</p>
Hashmaps and hashing algorithms</h2>
In 2017 Google introduced a new kind of hashmap for C++: the Swisstable
hashmap</a>. This structure uses SIMD
instructions to compute hashes, which is a great match for modern computer
hardware. So any improvements to the hashing algorithm would have an outsize
impact on performance, and in turn resource efficiency.</p>
In the cppcon 2017 talk on Swisstable</a>,
the authors mention that across all of Google's code, they found a non-trivial
amount of time was spent computing hashes 1</a></sup>.
Amanieu</a> from the libs team created a Rust variant
of the same algorithm for the
Hashbrown</a> crate, which has since
become the default Hashmap implemention for Rust.</p>
^{1</sup>
1% of CPU and 4% of RAM of all of Google's C++ code</em> is spent
doing things inside hashtables. Considering the scale of Google's deployment,
the amount of CPU cycles spent is enormous</em>. And it doesn't even consider
software running on user devices such as Android and Chrome, or programming
languages other than C++.</p>
</div>}However we've known for a long time about a faster hashmap than Swisstable:
the perfect hashtable</a>.
The idea is that if you know all your keys ahead of time, you can create a lookup
table which never creates collisions, never has to re-index the hashmap, or grow
in size2</a></sup>. Though it's not always possible to know all keys ahead of time, so
there are limits to when it can be used.</p>
^2</sup>There are a lot of considerations and tradeoffs for hashing
algorithms. Sy Brand gave a brief overview on
Twitter</a> in reply to
this post.</p>
</div>
Even so, let's take a look at when it does make sense to use perfect
hashtables, and show examples of how to implement them.</p>
enums as keys</h2>
Imagine we're building a protocol. The protocol has support for "metadata" and
"body data". The metadata consists of keys and values. We could imagine its use
could look something like this:</p>
use </span>std::collections::HashMap;
</span>
</span>let mut</span> metadata: HashMap<String, String> = HashMap::new();
</span>
</span>metadata.</span>insert</span>("</span>content_type</span>".</span>into</span>(), "</span>fast_proto</span>".</span>into</span>());
</span>metadata.</span>insert</span>("</span>encryption_key</span>".</span>into</span>(), generator.</span>next_key</span>());
</span>metadata.</span>insert</span>("</span>content_fingerprint</span>".</span>into</span>(), </span>bodyfingerprint</span>());
</span>metadata.</span>insert</span>("</span>content_length</span>".</span>into</span>(), </span>bodylength</span>().</span>into</span>());
</span>
</span>request.</span>send_metadata</span>(metadata).await?;
</span></code></pre>
We create a hashmap, insert several key-value pairs into it, and finally
serialize it over the wire. Now imagine these are the only</em> keys that are valid
to insert into the hashmap. Since we know all possible representations up
front, we could capture them within an enum:</p>
use </span>std::collections::HashMap;
</span>
</span>enum </span>KeyName {
</span>    ContentType,
</span>    EncryptionKey,
</span>    ContentFingerprint,
</span>    ContentLength,
</span>}
</span>impl </span>Display </span>for </span>KeyName { ... }
</span>
</span>let mut</span> metadata: HashMap<KeyName, String> = HashMap::new();
</span>
</span>metadata.</span>insert</span>(KeyName::ContentType, "</span>fast_proto</span>".</span>into</span>());
</span>metadata.</span>insert</span>(KeyName::EncryptionKey, generator.</span>next_key</span>());
</span>metadata.</span>insert</span>(KeyName::ContentFingerprint, body.</span>fingerprint</span>());
</span>metadata.</span>insert</span>(KeyName::ContentLength, body.</span>length</span>().</span>into</span>());
</span>
</span>request.</span>send_metadata</span>(metadata).await?;
</span></code></pre>
Because we're using an enum we know all of our keys ahead of time 3</a></sup>,
and can simply match</code> on the keys. Yet the code above will still expand to
roughly 1200 lines of assembly</a>. This is
because the Rust compiler currently isn't clever enough to lower the hashmap
into a lookup table.</p>
^{3</sup>
Yes yes, #[non_exhaustive]</code> would change that. But we aren't using
that here.</p>
</div>}Now what would happen if we had a "sufficiently smart compiler". The compiler
would be able to lower the hashmap to a fixed-sized array containing
Option<String></code> which uses match</code> to access the data within 4</a></sup>:</p>
^{4</sup>
We may not always want to use an array to store the data. But we're
dealing with a small key size here, so it's fine</em>.</p>
</div>}pub struct </span>HashMap {
</span>    </span>inner</span>: [Option<String>; 4],
</span>}
</span>
</span>impl </span>HashMap {
</span>    </span>pub fn </span>insert</span>(&</span>mut </span>self</span>, </span>key</span>: KeyName, </span>value</span>: String) -> Option<String> {
</span>        </span>match</span> key {
</span>            KeyName::ContentType => </span>self</span>.inner[</span>0</span>].</span>replace</span>(value),
</span>            KeyName::EncryptionKey => </span>self</span>.inner[</span>1</span>].</span>replace</span>(value),
</span>            KeyName::Fingerprint => </span>self</span>.inner[</span>2</span>].</span>replace</span>(value),
</span>            KeyName::ContentLength => </span>self</span>.inner[</span>3</span>].</span>replace</span>(value),
</span>        }
</span>    }
</span>
</span>    </span>pub fn </span>get</span>(&</span>self</span>, </span>key</span>: KeyName) -> Option<&String> {
</span>        </span>match</span> key {
</span>            KeyName::ContentType => </span>self</span>.inner[</span>0</span>].</span>as_ref</span>(),
</span>            KeyName::EncryptionKey => </span>self</span>.inner[</span>1</span>].</span>as_ref</span>(),
</span>            KeyName::Fingerprint => </span>self</span>.inner[</span>2</span>].</span>as_ref</span>(),
</span>            KeyName::ContentLength => </span>self</span>.inner[</span>3</span>].</span>as_ref</span>(),
</span>        }
</span>    }
</span>}
</span></code></pre>
What we're seeing here is that we've replaced the hashing function with a
match</code> statement. All data is stored within a fixed-sized array of Option</code>s,
and each enum variant directly maps to one of the slots within array. This
removes the overhead of hashing entirely, and opens the door for further
optimizations (such as inlining).</p>
Some might argue that: "We could just write this code ourselves", but
writing custom data structures has costs associated with it: we need to realize
we need the structure in the first place, then write the structure, maintain it,
and teach everyone else working on the codebase about it. Even though it might
be faster than Rust's default hashmap, going down this route has extra costs.</p>
static strings as keys</h2>
In many ways our example of keying into a hashmap using an enum is a bit too</em>
perfect. It assumes the author has awareness that their data is in fact
enumerable, and willingness to refactor the set of all keys into an enum. But it's also
risky: what if want to add more keys later. What if those keys may not be
known during compilation? I personally often prefer to keep things simple when
I'm not sure how things might change in the future.</p>
Code in the real world is often far less pristine. When trying to deliver a feature
under time pressure we're often more focused on getting it working and tested, than
on performance. After all it's usually fast enough</em> anyway. We'll often key into
hashmaps using strings, which have to be inlined into the created binary. The
assembly output for those inlined strings will look somewhat like this:</p>
.L__unnamed_1:
</span>        .ascii  "</span>content_type</span>"
</span>
</span>.L__unnamed_2:
</span>        .ascii  "</span>encryption_key</span>"
</span>
</span>.L__unnamed_3:
</span>        .ascii  "</span>content_fingerprint</span>"
</span>
</span>.L__unnamed_4:
</span>        .ascii  "</span>content_length</span>"
</span></code></pre>
A compiler not only knows which strings are being inlined. It also knows which
strings are used as inputs to which methods. This is to say: a compiler is very
much capable of capturing the full set of inputs during compilation 5</a></sup>, which is
provides us with data not unlike a Rust enum. That means at least theoretically
it should be possible to generate a perfect hashmap for the following input:</p>
^{5</sup>
Not sure if this currently occurs in LLVM or not. But I know I'm able
to obtain this data through Salsa in Rust-Analyzer.</p>
</div>}use </span>std::collections::HashMap;
</span>
</span>let mut</span> metadata: HashMap<&</span>'static str</span>, String> = HashMap::new();
</span>
</span>metadata.</span>insert</span>("</span>content_type</span>", "</span>fast_proto</span>".</span>into</span>());
</span>metadata.</span>insert</span>("</span>encryption_key</span>", generator.</span>next_key</span>());
</span>metadata.</span>insert</span>("</span>content_fingerprint</span>", body.</span>fingerprint</span>());
</span>metadata.</span>insert</span>("</span>content_length</span>", body.</span>length</span>().</span>into</span>());
</span>
</span>request.</span>send_metadata</span>(metadata).await?;
</span></code></pre>
And the generated perfect hashmap would be:</p>
pub struct </span>HashMap {
</span>    </span>inner</span>: [Option<String>; 4],
</span>}
</span>
</span>impl </span>HashMap {
</span>    </span>pub fn </span>insert</span>(&</span>mut </span>self</span>, </span>key</span>: &</span>str</span>, </span>value</span>: String) -> Option<String> {
</span>        </span>match</span> key {
</span>            "</span>content_type</span>" => </span>self</span>.inner[</span>0</span>].</span>replace</span>(value),
</span>            "</span>encryption_key</span>" => </span>self</span>.inner[</span>1</span>].</span>replace</span>(value),
</span>            "</span>content_fingerprint</span>" => </span>self</span>.inner[</span>2</span>].</span>replace</span>(value),
</span>            "</span>content_length</span>" => </span>self</span>.inner[</span>3</span>].</span>replace</span>(value),
</span>            _ => </span>unsafe </span>{ std::hint::unreachable_unchecked() },
</span>        }
</span>    }
</span>    
</span>    </span>pub fn </span>get</span>(&</span>self</span>, </span>key</span>: &</span>str</span>) -> Option<&String> {
</span>        </span>match</span> key {
</span>            "</span>content_type</span>" => </span>self</span>.inner[</span>0</span>].</span>as_ref</span>(),
</span>            "</span>encryption_key</span>" => </span>self</span>.inner[</span>1</span>].</span>as_ref</span>(),
</span>            "</span>content_fingerprint</span>" => </span>self</span>.inner[</span>2</span>].</span>as_ref</span>(),
</span>            "</span>content_length</span>" => </span>self</span>.inner[</span>3</span>].</span>as_ref</span>(),
</span>            _ => </span>unsafe </span>{ std::hint::unreachable_unchecked() },
</span>        }
</span>    }
</span>}
</span></code></pre>
hybrid static + dynamic keys</h2>
If our keys are only ever known during runtime, we know the best hashmap we can
use is Hashbrown</code>. But what if some of our keys are known during compilation?</p>
This is where we can use a "hybrid" hashmap 6</a></sup>: we generate a lookup table for
the known keys, and use a regular hashmap as a fallback for all other keys. This
approach is used by hyperium/http</code></a>'s
HeaderMap</code> struct. It keeps track of commonly used HTTP headers in a lookup
table, and falls back to a hashmap for all other headers.</p>
^{6</sup>
I made up the term "hybrid hashmap" for the purpose of this post. Let me know
if you know of a term used in literature to describe this same structure.</p>
</div>
Here's a usage example of a hashmap which uses both dynamic and static data:</p>}use </span>std::collections::HashMap;
</span>
</span>let mut</span> metadata: HashMap<String, String> = HashMap::new();
</span>
</span>metadata.</span>insert</span>("</span>content_type</span>".</span>into</span>(), "</span>fast_proto</span>".</span>into</span>());
</span>metadata.</span>insert</span>("</span>encryption_key</span>".</span>into</span>(), generator.</span>next_key</span>());
</span>metadata.</span>insert</span>(</span>gen_keyname</span>(), </span>gen_headername</span>()); </span>// this generates values at runtime
</span>
</span>request.</span>send_metadata</span>(metadata).await?;
</span></code></pre>
For this code we're able to generate the following implementation. We create
a lookup table for our known keys, and fall back to a hashmap for all other keys:</p>
pub struct </span>HashMap {
</span>    </span>inner</span>: [Option<String>; 4],
</span>    </span>map</span>: std::collections::HashMap<String, String>,
</span>}
</span>
</span>impl </span>HashMap {
</span>    </span>pub fn </span>insert</span>(&</span>mut </span>self</span>, </span>key</span>: String, </span>value</span>: String) -> Option<String> {
</span>        </span>match</span> key.</span>as_ref</span>() {
</span>            "</span>content_type</span>" => </span>self</span>.inner[</span>0</span>].</span>replace</span>(value),
</span>            "</span>encryption_key</span>" => </span>self</span>.inner[</span>1</span>].</span>replace</span>(value),
</span>            "</span>content_fingerprint</span>" => </span>self</span>.inner[</span>2</span>].</span>replace</span>(value),
</span>            "</span>content_length</span>" => </span>self</span>.inner[</span>3</span>].</span>replace</span>(value),
</span>            _ => </span>self</span>.map.</span>insert</span>(key, value),
</span>        }
</span>    }
</span>
</span>    </span>pub fn </span>get</span>(&</span>self</span>, </span>key</span>: &</span>str</span>) -> Option<&String> {
</span>        </span>match</span> key {
</span>            "</span>content_type</span>" => </span>self</span>.inner[</span>0</span>].</span>as_ref</span>(),
</span>            "</span>encryption_key</span>" => </span>self</span>.inner[</span>1</span>].</span>as_ref</span>(),
</span>            "</span>content_fingerprint</span>" => </span>self</span>.inner[</span>2</span>].</span>as_ref</span>(),
</span>            "</span>content_length</span>" => </span>self</span>.inner[</span>3</span>].</span>as_ref</span>(),
</span>            _ => </span>self</span>.map.</span>get</span>(key),
</span>        }
</span>    }
</span>}
</span></code></pre>
A nice property about this approach is that if we only ever use known keys, we'll get
speeds similar to that of a lookup table. It's only once we start using custom keys
that we branch into a slower path and start allocating for the hashmap.</p>
Looking ahead</h2>
While it's possible for humans to look for instances where a hashmap can be
converted into a lookup table, for Rust only a compiler will be able to catch
all</em> cases. This is because virtually no code in Rust is ever completely
self-contained.</p>
Say we have a hashmap defined in crate A. And we have a crate B which
uses that hashmap. If B only ever uses A with static keys we're able to optimize
for that use. However crate A will cannot know ahead of time what the usage
patterns of B will be like.</p>
Add in more crates, transient wrappers, and legacy code - and it quickly becomes
clear why it's ideal to let machines handle these kinds of optimizations.
However the question then becomes: how would we enable this in Rust?</p>
I think if we had access to the following information during const /
specialization we'd be able to create these kinds of optimizations:</p>

Provide a way to enumerate all static inputs for a function.</li>
Provide a way to tell if there are runtime inputs for a function.</li>
Provide a way to count the number of invocations a function has within the
codebase.</li>
</ol>
All of this information can be known by programmers ahead of time by reading
the code they're working on. This means that a compiler can know this
information too. The question really is how (and where) to expose this
information.</p>
My gut feeling is that we should be able to figure this out somewhere during
const evaluation and specialization 7</a></sup> 2</a></sup>. Using an entirely made-up,
fantasy pseudo version of Rust I'd imagine we'd be able to create a new hashmap
using something like this:</p>
^{7</sup>
Yes I know that's not a separate step, - less of a time, more of a place I guess.</p>
</div>}^{2</sup>
I don't believe we can just specialize every hashmap + hasher combo.
I'm thinking we might be able to specialize Rust's default hashmap though. As I understand it we
don't guarantee much of anything other than it being fast and not being a DDoS
vector.</p>
</div>}#[</span>this_is_a_specialization</span>]
</span>const fn </span>create_hashmap</span><K: </span>'static</span>, V>(</span>m</span>: Map<K, V>) -> HashMap {
</span>    </span>// how often is this map's `get` function called?
</span>    </span>if </span>mem::invocation_count_of(m.get) == </span>0 </span>{
</span>        </span>// return stub structure because we know the data is never accessed
</span>    }
</span>
</span>    </span>// iterate over all static inputs provided to m.insert
</span>    </span>for </span>(k, _) in mem::static_inputs_of(m.insert) {
</span>        </span>// iterate over all keys and return a lookup table
</span>    }
</span>
</span>    </span>// count how often m.insert is called with non-static input
</span>    </span>if </span>mem::dynamic_input_count_of(m.insert) != </span>0 </span>{
</span>        </span>// if there are runtime (non-static) arguments we want to return a hybrid hashmap
</span>    } </span>else </span>{
</span>        </span>// yay, return a lookup table
</span>    }
</span>}
</span></code></pre>
Maybe there are other proposals in progress for const execution / specialization
which cover these cases as well. Admittedly adding a bunch of new APIs backed by
/* compiler built-in */</code> is a bit of a cop-out, and I'd love to learn if there
could be ways to do the same which more closely integrate with what we already
have.</p>
I hope this paints a rough picture of what I'm thinking. I wish I knew more
about the present and future of Rust's const engine to design a better API. But
my goal with this post is less to propose something concrete, and more to convince
the right people this is something worth exploring. I hope in that regard
I'm doing alright so far (:</p>
Conclusion</h2>
I hope everyone walks away from this post with two things: an understanding of
how to create a lookup table/perfect hashmap in Rust, and excitement about a future
where our compilers can apply these optimizations for us.</p>
The idea of transparently optimizing existing patterns based on usage is
fascinating to me. Usually when I think about these kinds of optimizations it's
things like compilers optimizing math by converting it to bitwise instructions.
But imagine how cool it would be if we could optimize entire data structures
based on how they're used. This type of optimization doesn't need to stop with
hashmaps either: I got interested in this topic because I think we can optimize
channels significantly if we can inspect their use during compilation
8</a></sup>.</p>
^{8</sup>
For example: we could optimize an MPMC channel into a one-shot SPSC
channel if we know both the sender and receiver are never cloned, and only one
item is ever sent.</p>
</div>
Another thing which excites me about this is that if we're able to lower a
hashmap into a match</code> statement, we can proceed to apply all optimizations we
use to match statements. This means optimizations such as removing unused enum
variants, reordering keys for quicker lookups, and even lowering the match
statement itself to more efficient structures such as prefix-tries. Rust's
match</code> performance is probably not where it could be yet either, and the more
we can express code of one kind in terms of another, the more impactful
optimizations can become.</p>
Thanks for reading!</p>}

refcounts
Fri, 07 May 2021 00:00:00 +0000
MyData I've wondered about for a while now is how to improve the ergonomics
of some of the structures in std::sync</code>. There's quite a bit of excitement
gathering about ghost-cell</code></a>
(and the paper</a>), and I figured
this would be as good a time as any to share some thoughts (questions?) on
ergonomics.</p>
Like my last post, the intent of this post is less to provide a solution or
result of research, but instead share my current thinking on the topic. Rather
than asking questions in private, I figured I'd ask questions on here instead (:</p>
std::sync::Barrier</code></h2>
Let's use
std::sync::Barrier</code></a>
to center our examples around. This is a data structure which enables synchronization between
any number of threads. One example of how you might want to use this is in an
HTTP server. You may want to create a connection to a database, connect to a
cache server, connect to perhaps some other backend, and only when all of those
have succeeded do you proceed to accept connections. If one of them takes a
while to initialize (or even fails entirely), our end-users won't be impacted
because none of their requests are being handled yet.</p>
This structure enables multiple threads to synchronize</em> before starting work.
Its use usually looks something like this:</p>
use </span>std::sync::{Arc, Barrier};
</span>use </span>std::thread;
</span>
</span>let mut</span> handles = Vec::with_capacity(</span>10</span>);
</span>let</span> barrier = Arc::new(Barrier::new(</span>10</span>));
</span>for </span>_ in </span>0</span>..</span>10 </span>{
</span>    </span>let</span> c = Arc::clone(&barrier);
</span>    </span>// The same messages will be printed together.
</span>    </span>// You will NOT see any interleaving.
</span>    handles.</span>push</span>(thread::spawn(</span>move</span>|| {
</span>        println!("</span>before wait</span>");
</span>        c.</span>wait</span>();
</span>        println!("</span>after wait</span>");
</span>    }));
</span>}
</span>// Wait for other threads to finish.
</span>for</span> handle in handles {
</span>    handle.</span>join</span>().</span>unwrap</span>();
</span>}
</span></code></pre>
We create an instance of Barrier</code> by passing it a count, wrap it in an Arc</code>
(so it can be cloned), and then pass an instance of it to each thread. Important
to note is that the number of threads we spawn needs to exactly</em> line up to the
number we've apssed into Barrier::new</code>. If the number is too high our
application will lock up and never continue. If the number is too low, the
application will start before all threads are ready, defeating the point of
using the structure in the first place.</p>
dynamic Barrier</h2>
One difficulty with std::sync::Barrier</code> is that we need to statically</em> pass
the number in. This means in order to create the synchronization structure we
need to know up front how many items we're trying to synchronize. Say we're
dealing with an iterator of unknown length, and we want to exhaust all items
before continuing? We don't know the number up front, so we can't use the
structure:</p>
use </span>std::thread;
</span>use </span>std::sync::{Arc, Barrier};
</span>
</span>fn </span>from_iter</span>(</span>iter</span>: impl Iterator<Item = MyData>) {
</span>    </span>let mut</span> handles = Vec::new();
</span>    </span>let</span> barrier = Arc::new(Barrier::new(</span>/* ??? */</span>));
</span>    </span>for</span> item in iter {
</span>        </span>// ... create a thread and wait within
</span>    }
</span>    </span>// ... join all handles
</span>}
</span></code></pre>
Counting at runtime is not something exceptional though. In our example Arc</code>
works entirely at runtime, making work without passing a count ahead of time.
Though we don't have a dynamic version of Barrier</code> in the stdlib, we do</em> have
one in the ecosystem in the form of:
crossbeam::sync::WaitGroup</code></a>.
With it we can rewrite our previous example like so:</p>
use </span>std::thread;
</span>use </span>crossbeam::sync::WaitGroup;
</span>
</span>fn </span>from_iter</span>(</span>iter</span>: impl Iterator<Item = MyData>) {
</span>    </span>let mut</span> handles = Vec::new();
</span>    </span>let</span> wg = WaitGroup::new();    </span>// 1. create a new instance.
</span>    </span>for</span> item in iter {
</span>        </span>let</span> wg = wg.</span>clone</span>();      </span>// 2. clone it
</span>        handles.</span>push</span>(thread::spawn(</span>move</span>|| {
</span>            println!("</span>before wait</span>");
</span>            wg.</span>wait</span>();            </span>// 3. wait to sync
</span>            println!("</span>after wait</span>");
</span>        }));
</span>    }
</span>
</span>    wg.</span>wait</span>();                    </span>// 4. also sync our original wg instance
</span>
</span>    </span>// ... join all handles
</span>}
</span></code></pre>
The upside of using WaitGroup</code> over Barrier</code> is that it's much easier to use,
and more widely applicable. The downside is that it's (marginally) less efficient
since we need to count the instances at runtime rather than having that
information up front 1</a></sup>.</p>
^{1</sup>
One could argue that manually passing the count up front can be an
extra safeguard - but there's inherent risks to mis-counting the number of
instances. So I'm not sure that's much of an extra safeguard. More on that later.</p>
</div>
I don't want to argue for including WaitGroup</code> in the stdlib since it's unclear
how common the use of Barrier</code> is. But I hope to have illustrated that
Barrier</code> has limitations in how it can be used to create cross-thread
synchronization points.</p>}const Barrier</h2>
As we've seen in the first example there's another issue: we need to ensure that
the number passed to Barrier::new</code> matches the number of threads we spawn. One
way we could fix this is by creating a variable, and passing that same variable
to both the loop and Barrier::new</code>:</p>
let</span> count = </span>10</span>;
</span>let mut</span> handles = Vec::new();
</span>let</span> barrier = Arc::new(Barrier::new(count));
</span>for </span>_ in </span>0</span>..count {
</span>    </span>let</span> c = Arc::clone(&barrier);
</span>    </span>/* ... */
</span>}
</span></code></pre>
This definitely works, but it still manually requires synchronizing the value in
Barrier</code> and the value in the loop. Even though it's more efficient than using
WaitGroup</code>, it's less ergonomic to use. However if we, as humans, look at the
code it's immediately clear what the value in Barrier::new</code> should be. Namely:
we're calling Arc::clone</code> ten times, so we know ahead of time</em> that the value
we pass to Barrier::new</code> should be 10</code>.</p>
Now here's what I wonder (and honestly, am not sure about). Given the number of
clones can be statically inferred by humans by looking at the code, could we
find a way to for this information to become statically available during const</code>?</p>
let</span> count = </span>10</span>;
</span>let mut</span> handles = Vec::new();
</span>let</span> barrier = Barrier::new()
</span>for </span>_ in </span>0</span>..count {
</span>    </span>let</span> c = barrier.</span>clone</span>();
</span>    </span>/* ... */
</span>}
</span></code></pre>
That would allow us to create a dynamic barrier and compile-time barrier with
similar APIs, but different assurances. The dynamic barrier would be more
flexible and work for unknown inputs, and the compile-time barrier would be more
efficient, but only work if the number of items is known at compile-time.</p>
Conclusion</h2>
Using std::sync::Barrier</code> only acts an example. The same reasoning could be
applied to e.g. Arc</code>, Rc</code>, channel</code>, and more. For example for channels it
could be interesting if we could compute ahead of time what the max number of
items that might be in flight are, and fill that in as the queue size. Caveats
and circumstances apply, but the general premise is: "If some item count is
known at compile time, is there a way we can pass that number back to our
structures?"</em></p>
Crates such as static_rc</code></a> are
getting close to this idea, but still require manually passing in the count. I'm
wondering whether it could be possible to eliminate that count entirely.</p>
edit</strong>: I found this
thread</a>
by the author of static_rc</code> which asks very similar questions as we do here.
Also lcnr shared some thoughts on Twitter about the feasibility</a> of what refcounting in const.</p>
Before closing out this post, I want to re-iterate once more that nothing of
this is a concrete proposal. I'm not even advocating for including
crossbeam::sync::WaitGroup</code> in the stdlib. The purpose of this post is to share
thinking, and ask questions about what is desireable and possible within the
(future of) the Rust language.</p>
And with that, thanks so much for reading this - happy weekend!</p>

Futures Concurrency Notes: join and const-eval
Sat, 16 Jan 2021 00:00:00 +0000
Happy new year everyone! Today we're trying something new: less of a blog
post, more of research notes</em>. This is less of a "here's something I've
concluded", and more of: "here's something I'm thinking about". Today's topic
is: "How can we add Future::{try_}join</code> and {try_}join!</code> to the stdlib in a
way that feels consistent?"</p>
What does joining Futures do?</h2>
A Future in Rust is best though of as a "value which eventually becomes
available". It's not specified when</em> a value becomes available, so using
.await</code> allows us to wait for it until it's available.</p>
Sometimes we want to wait on more than one future at the time: after all,
when we're waiting on things, we can do other things in the mean time. And
one way to do this is by calling join</code>.</p>
async-std</code> exposes a Future::join</code> method, and async-macros</code> exposes a
join!</code> macro. An example joining two futures:</p>
let</span> a = future::ready(</span>1</span>u8</span>);
</span>let</span> b = future::ready(</span>2</span>u8</span>);
</span>assert_eq!(join!(a, b).await, (</span>1</span>, </span>2</span>));
</span>
</span>let</span> a = future::ready(</span>1</span>u8</span>);
</span>let</span> b = future::ready(</span>2</span>u8</span>);
</span>assert_eq!(a.</span>join</span>(b).await, (</span>1</span>, </span>2</span>));
</span></code></pre>
However once we start joining more than two futures, the output types become
different:</p>
let</span> a = future::ready(</span>1</span>u8</span>);
</span>let</span> b = future::ready(</span>2</span>u8</span>);
</span>let</span> c = future::ready(</span>3</span>u8</span>);
</span>assert_eq!(join!(a, b, c).await, (</span>1</span>, </span>2</span>, </span>3</span>));
</span>
</span>let</span> a = future::ready(</span>1</span>u8</span>);
</span>let</span> b = future::ready(</span>2</span>u8</span>);
</span>let</span> c = future::ready(</span>3</span>u8</span>);
</span>assert_eq!(a.</span>join</span>(b).</span>join</span>(c).await, (</span>1</span>, (</span>2</span>, </span>3</span>)));
</span></code></pre>
As you can see, each invocation of Future::join</code> returns a tuple. But that
means that chaining calls to it starts to nest tuples, which becomes hard to
use. And it becomes more nested the more times you chain. Oh no!</p>
In contrast, the join!</code> macro dynamically grows the number of items returned
in the tuple. This is possible because macros have loops, and can just write
code -- so inside the macro we just expand the output to a tuple which is
large enough to hold all of the outputs.</p>
What does join!</code> return?</h2>
The definition of async_macros::join!</code> is fairly brief, so I'll just share
it right here. The only detail missing is the defition of the MaybeDone</code>
type: it's a wrapper which can be awaited, and stores the output type of the
future once it completes. We wait for all instances of MaybeDone</code> to
complete, and at the end we take all their values and return it from the
future:</p>
#[</span>macro_export</span>]
</span>macro_rules! </span>join {
</span>    ($(</span>$fut</span>:</span>ident</span>),* $(,)?) => { {
</span>        async {
</span>            $(
</span>                </span>// Move future into a local so that it is pinned in one place and
</span>                </span>// is no longer accessible by the end user.
</span>                </span>let mut </span>$fut </span>= </span>$crate</span>::MaybeDone::new(</span>$fut</span>);
</span>            )*
</span>            </span>$crate</span>::utils::poll_fn(</span>move </span>|cx| {
</span>                </span>use </span>$crate</span>::utils::future::Future;
</span>                </span>use </span>$crate</span>::utils::task::Poll;
</span>                </span>use </span>$crate</span>::utils::pin::Pin;
</span>
</span>                </span>let mut</span> all_done = </span>true</span>;
</span>                $(
</span>                    </span>let</span> fut = </span>unsafe </span>{ Pin::new_unchecked(&</span>mut </span>$fut</span>) };
</span>                    all_done &= Future::poll(fut, cx).</span>is_ready</span>();
</span>                )*
</span>                </span>if</span> all_done {
</span>                    Poll::Ready(($(
</span>                        </span>unsafe </span>{ Pin::new_unchecked(&</span>mut </span>$fut</span>) }.</span>take</span>().</span>unwrap</span>(),
</span>                    )*))
</span>                } </span>else </span>{
</span>                    Poll::Pending
</span>                }
</span>            }).await
</span>        }
</span>    } }
</span>}
</span></code></pre>
As you can see the outer-most value returned is an async {}</code> block. This
isn't a specific type, but can be referred to using impl Future</code>. However
the type returned by Future::join</code> is a concrete Join</code> future. This type
can</em> be addressed by name, and actually passed around.</p>
However as we chain Future::join</code> repeatedly, the resulting future's
signature will look somewhat like: Join<Join<Join<Join<T>>>></code>. This is not
great.</p>
So on the one hand we have anonymous futures which can only be addressed
through impl Future</code>. And on the other hand we have deeply nested futures
which are a pain to write by hand. Can we do better?</p>
consistent return types</h2>
One thing I mentioned at the start but didn't dive in yet is the fact that
we'd like to align the return types of join!</code> and Future::join</code>. Even if
Future::join</code> would only ever take one other future as an argument, being
able to switch between the method and the macro without needing to change the
signature of the returned types is a a huge bonus.</p>
After having worked on async-std</code> for the past two years where a lot of APIs
use async fn</code>, I'm now somewhat convinced that the stdlib should never</em> do
this. Which you can see reflected in APIs such as std::future::ready</code> which
now returns the concrete future std::future::Ready</code>, whereas in async-std</code>
it was just an async fn</code>.</p>
Probably another point worth touching on is the futures-rs</code> implementation
of join!</code>. This doesn't return any kind of future at all, wrapping the
.await</code> call within the macro instead. I feel somewhat strongly that
.await</code> calls shouldn't be hidden in code, but instead always be visible.</p>
// Example of futures_rs::join!
</span>let</span> a = future::ready(</span>1</span>u8</span>);
</span>let</span> b = future::ready(</span>2</span>u8</span>);
</span>assert_eq!(join!(a, b), (</span>1</span>, </span>2</span>)); </span>// is this sync or async?
</span></code></pre>
This slight digression into futures-rs</code> aside, I think if we were to add
future::Future::join</code> and future::join!</code> functions to the stdlib, both
should be returning concrete futures. And because they effectively do the
same thing, we should make it so they both return the same Join</code> type.</p>
Maybe const can help?</h2>
So the question now becomes: "how can we do this?". And I think the answer
for this is: "const tuples may be able to help".</p>
So const tuples don't exist in Rust today yet. Not even on nightly. The only
way to create variadic tuples is through macros like we've shown. However
from talking to members of the const-eval WG const tuples are definitely on
the roadmap, though it may take a while. However now that we're seeing a move
to fund more people to work on the compiler, I'm hoping that this may be
possible within a few years, which isn't that</em> long in the grand scheme of
things.</p>
Given there's no proposal for const tuples, it's hard to write an example
since I have no clue what the syntax for it will be. For N-length arrays the
syntax is the following:</p>
pub fn </span>array_windows</span><</span>const</span> N: </span>usize</span>>(&</span>self</span>) -> ArrayWindows<'_, T, N>;
</span></code></pre>
The const N: usize</code> here is the length argument for the array of type T</code>.
The operations this function returns are on [T; N]</code>. However tuples don't
have a consistent type T</code>; values contained within tuples are heterogenous.
So a tuple of length N</code> can contain N</code> different types. I have no clue how
this would be expressed in const</code> contexts (if at all possible?).</p>
So for now let's just pretend we can define N-length tuples inside</em> function
signatures, and cross our fingers that this makes enough sense that the idea
comes across. Assuming something like that would work, I would expect
Future::Join</code> to be able to defined along these lines:</p>
impl </span>Future {
</span>    </span>/// Join with one other future.
</span>    </span>pub fn </span>join</span><F: Future>(</span>self</span>, </span>other</span>: F) -> Join<(</span>Self</span>, F)>;
</span>}
</span></code></pre>
This signature tries to convey: this future holds at least two futures:
Self</code>, and another future we're joining with. For the join!</code> macro we could
fill out the types using code generation, populating the values of the tuple
at compile time. Invoking it would yield the following return type:</p>
let</span> a = future::ready(</span>1</span>u8</span>);
</span>let</span> b = future::ready(</span>2</span>u8</span>);
</span>let</span> fut = join!(a, b);
</span>// fut: Join<(Ready<Output = u8>, Ready<Output = u8>)>
</span>
</span>let</span> a = future::ready(</span>1</span>u8</span>);
</span>let</span> b = future::ready(</span>2</span>u8</span>);
</span>let</span> c = future::ready(</span>3</span>u8</span>);
</span>let</span> fut = join!(a, b, c);
</span>// fut: Join<(Ready<Output = u8>, Ready<Output = u8>, Ready<Output = u8>)>
</span></code></pre>
Assuming once we have const tuples we'll also have const panic, we can guard
against the case where zero futures are provided to Join</code>. Or perhaps the
signature should instead be:</p>
Join<</span>Self</span>, (Other, Other2)>
</span></code></pre>
The details are unclear, because well, I don't know how this should work in
the future. Maybe there's a different feature a play here too: what if
expressing this in signatures actually requires const-variadics</code> or
something. This may actually be relying on a variety of features I'm not
tracking.</p>
What does that mean for adding futures concurrency to the stdlib?</h2>
This post is rooted in research I was doing exactly to answer that question.
join!</code> and Future::join</code> feel like they do exactly what they should, module
some issues around their return types. Unfortunately however it seems the
best solution would require a const-eval feature that doesn't even have an
RFC yet.</p>
Given I expect Rust to stick around for at least a few more decades, and how
core this functionality is for async programming. I think it's actually worth
waiting to implement these features correctly, rather than rushing to add
them in the short term. Libraries such as async-std</code> and async-macros</code> can
provide suitable solutions through user space in the interim.</p>
In addition to that, there's one more feature in the language required before
we can consider adding Future::join</code>: namely, we need either
#[cfg(accessible)]</code></a>, or
#[cfg(version)]</code></a>. This is
currently a blocker to adding any</em> method on the Future</code> trait. Since the
majority of the async ecosystem relies on Ext</code> traits to implement missing
functionality, adding a method of the same name to the stdlib would cause
ambiguity. So in order to prevent accidental ecosystem breakage, libraries
outside of std should gain the ability to detect whether a method has been
implemented in the stdlib. Which is what accessible</code> and version</code> are for.</p>
However one possibility may be that we add join!</code> based on the async {}</code>
block implementation in the near term without adding Future::join</code> as well.
That would at least allow us to expose that funcionality from the stdlib,
even if in some instances it may not be the most ergonomic.</p>
Then later on, once we gain the ability to reason about tuples/variadics in
const contexts, we can switch the return type to be Join</code>, and add the
Future::join</code> method as well. That way we get a solution in the short term,
but still do the right thing in the long term. This depends on async {}</code>
being forward compatible with returning a concrete future though. I'm not
sure if this has been done before, and the lang team might need to weigh in
on that.</p>
edit (2020-01-17):</strong> As pointed out here</a>
by matthieum, if we had a variadic Join</code> type, there's no reason we couldn't
implement join</code> on tuples directly.</p>
// Tuple::join
</span>let</span> a = future::ready(</span>1</span>u8</span>);
</span>let</span> b = future::ready(</span>2</span>u8</span>);
</span>let</span> c = future::ready(</span>3</span>u8</span>);
</span>assert_eq!((a, b, c).await, (</span>1</span>, </span>2</span>, </span>3</span>));
</span>
</span>// Array::join
</span>let</span> a = future::ready(</span>1</span>u8</span>);
</span>let</span> b = future::ready(</span>2</span>u8</span>);
</span>let</span> c = future::ready(</span>3</span>u8</span>);
</span>assert_eq!([a, b, c].await, [</span>1</span>, </span>2</span>, </span>3</span>]);
</span></code></pre>
This would soft-deprecate the need to use future::join!</code> and could probably
also be extended to arrays and slices of futures too 1</a></sup>. The question is
whether the same Join</code> type could be used for all implementations, since it
wouldn't return a tuple but an array or vec instead. This can probably only
be answered once designs for the corresponding language features start.</p>
^1</sup>join!</code> on an array is effectively an instance of
join_all!</code></a>.
In my designs I've mostly relegated join_all!</code> as not being a primitive,
instead favoring designs such as TaskGroup</code> and ParallelStream</code> for a
collection of N futures since these more often than not will want to be run
on a executor anyway. However wanting to join N futures is still nice to be
able to do, and implementing Array::join</code> may very well provide a way for us
to do so.</p>
</div>
Other considerations</h2>
Everything we've expressed here not only applies to future::Future::Join</code>
and future::join!</code>. It applies to the try_join</code>, race</code>, and try_race</code>
variants as well. The
async_std::future</code></a>
docs explain how these types cover the full range of awaiting futures.</p>
Additionally the future::join</code> variants only really work well when you know
ahead of time how many futures you're going to be awaiting. If the number is
dynamic, other constructs should be used. In a future post I may talk about
TaskGroup</code>, an adaptation of crossbeam::scope</code> I'm working on, inspired by
Swift's upcoming task</code> proposal. But other constructs like parallel-stream</code>
and FuturesUnordered</code> already exist as well.</p>
Conclusion</h2>
In this post we've looked at what it would take to add Future::join</code> and
join!</code> to the stdlib where both functions would return the same, named
future. One plausible way to achieve this would be through const tuples (and
possible const variadics, which may or may not be the same thing).</p>
However it may be possible to add future::join!</code> in the near term, and once
Rust gains the appropriate language features add Future::join</code> and upgrade
join!</code> to use the same Join</code> future. This would enable adding the
functionality in the near term, but still achieving the ideal design later
on.</p>
This post is a bit of an experiment: single draft, Saturday morning writing.
I've been doing a lot of research into stabilizing async Rust paradigms
recently, and figured I'd share some of the findings along the way. In part
for my own reference. But also to communicate needs async Rust may have to
members of other Rust teams.</p>
Anyway, hope you enjoyed this, and hope you have a good weekend!</p>
If you or your company like the work I'm doing, you're welcome to support
me through GitHub sponsors</a>. Special thanks
to my sponsors: hibbian</a>,
milesgranger</a>,
romatthe</a>, No9</a>, and
several others who prefer to remain anonymous.</em></p>

Rust 2021
Thu, 24 Sep 2020 00:00:00 +0000
I am tired. Today was rough. This year has been rough. Everyone around me is
tired too. 2020 has been a hot mess, coming right after a not so hot 2019,
and a messy 2018. It's not just you. It's everything. It's okay.</p>
My wish for Rust 2021 is for you to go easy on yourself. No really. You. If
you've done any work on Rust at all, that means you doubly so. Yes
really</em>.</p>
The core team has tasked themselves with starting a
foundation</a>.
Laudible, but can we take a sec to realize we're still mid-pandemic. Not that
long ago our biggest sponsor cut back drastically. People lost their jobs.
Co-workers. Friends. People we love and cherish. Things are tough. And
there's still much</a>,
much</a>, much more.
And right while all this is happening we're creating this foundation. Ooph.</p>
That's why I want the core team to take a break. Go on a vacation. Close the
office for a while. I don't know when. Maybe do one in the winter. Maybe one
in summer, after we have a foundation. Big ol' summer vacation. Too hot to
think anyway.</p>
Actually not just the core team: I want everyone to take a vacation. The
whole Rust project. Hit the brakes, take it easy. Archive rust-lang/rust</code>
for a week. Can't come to work if the office is locked. Take a breath. Take a
bath.</p>
"Rust" is the outcome of the processes we apply. It's through dialog, labor,
and mutual respect that we get any of this done. And we can't do these things
if we can't talk about how we feel. And uhh, reading the room here, but
people don't seem to be doing well. How can we feel safe in dialog if we
don't even feel physically safe. We really gotta take care of ourselves first
before we take care of RFC2000.</p>
Of course we also should find ways to get
paid</a>. Of course
stdsimd</a> is cool, and
error
handling</a>
is cool, and async</a> is
cool, and…</p>
Yea.</p>
The backlog of things to work on is infinite. The people that work on them
aren't. The cool problems to work on will be here when we come back. I just
hope you'll still be here with me ♥</p>

Async Iteration I: Iteration Semantics
Fri, 18 Sep 2020 00:00:00 +0000

This post is part of the Async Iteration series:</em></p>

Async Iteration I: Semantics</a> (this post)</li>
Async Iteration II: Async Iterator Crate</a></li>
Async Iteration III: Async Iterator Trait</a></li>
</ol>

In a recent lang team meeting</a> we discussed
the upcoming
Stream</code></a>
RFC. We covered both the steps required to land it, and the steps we'd want
to take after that. One of the things we'd want eventually for streams is:
"async iteration syntax". Just like for x in y</code> works for Iterator</code>, we'd
want something similar to work for Stream</code>. For example JavaScript support
both
sync</a>
and
async</a>
iteration through for..of</code> loops:</p>
// sync
</span>for </span>(</span>let </span>num </span>of </span>iter</span>) {
</span>  </span>console</span>.</span>log</span>(</span>i</span>);
</span>}
</span>
</span>// async
</span>for await </span>(</span>let </span>event </span>of </span>emitter</span>) {
</span>    </span>console</span>.</span>log</span>(event);
</span>}
</span></code></pre>
However when talking about async iteration syntax in Rust one question has
come up repeatedly:</p>

Should async iteration be parallel or sequential by default?</p>
</blockquote>
In this post I'd like to cover the challenges I see with parallel async
iteration, and make the case for, spoilers, defaulting to sequential
iteration semantics.</p>

However before we continue, the usual disclaimers: I'm not on the lang
team. I do not hold any direct decision making power about the Rust language.
I am a member of the Async Foundations WG, but I do not speak for anyone on
the team other than myself.</strong> This post is from my perspective only.</p>
The problem space</h2>
So the core question we're trying to answer is: "should async iteration
default to sequential or parallel?" Putting this into code, it's a question
of what some hypothetical future "async iteration" syntax should compile down
to, and which semantics are provided:</p>
// This code...
</span>for</span> event.await in channel {
</span>    dbg!(event);
</span>}
</span>
</span>// ... should it lower into this? (sequential)
</span>while let </span>Some(event) = channel.</span>next</span>().await {
</span>    dbg!(event);
</span>}
</span>
</span>// ... or should it lower into this? (parallel)
</span>channel.</span>into_par_stream</span>().</span>for_each</span>(async |event| dbg!(event));
</span></code></pre>
The "lowering" we're describing here is not the precise</em> code we'd expect
the compiler to output. But hopefully it gets the point across. In the
"sequential" case we'd expect async iteration loops to behave much the same
as async while let Some()</code> loops do today. And in the parallel case, we'd
expect something to work with semantics similar to those of
parallel-stream</a>.</p>
The main argument I've heard in favor of parallel async iteration semantics
is that in most cases it's what people want anyway. For example take the
following code:</p>
use </span>async_std::io;
</span>use </span>async_std::net::TcpListener;
</span>use </span>async_std::prelude::*;
</span>
</span>let</span> listener = TcpListener::bind("</span>127.0.0.1:8080</span>").await?;
</span>
</span>for</span> stream.await? in listener.</span>incoming</span>() {
</span>    io::copy(&stream, &stream).await?;
</span>}
</span></code></pre>
Under sequential iteration semantics this loop would process at most 1 TCP
connection at the time, irregardless of system resources available. But under
parallel iteration semantics, this will have access to all resources
available. The idea is that parallel iteration semantics provide a language
level solution to this problem.</p>
Control flow operators</h2>
The first issue that arises with parallel async iteration is that the break</code>
and continue</code> control flow operators will have different semantics, or need
to be disallowed altogether. Take the following code:</p>
use </span>async_std::io;
</span>use </span>async_std::net::TcpListener;
</span>use </span>async_std::prelude::*;
</span>
</span>let</span> listener = TcpListener::bind("</span>127.0.0.1:8080</span>").await?;
</span>
</span>// Let's pretend this is valid syntax..
</span>for </span>(i, stream?).await in listener.</span>incoming</span>().</span>enumerate</span>() {
</span>    </span>if</span> i === </span>1000 </span>{
</span>        </span>break</span>;
</span>    }
</span>    io::copy(&stream, &stream).await?;
</span>}
</span></code></pre>
Say we have a few dozen parallel requests open when we hit break</code>. We should
probably stop accepting new requests. What should happen to the still open
connections?</p>
There is no right answer. We may want to drop them. We may want to keep them
running. More likely the desired behavior may depend on what you're doing.
But the semantics will always</em> be different than in regular for loops
because</em> of parallelism.</p>
Adding new semantics to a well-established concept is tricky, and may
actually lead to subtle translation issues when converting existing
algorithms to async Rust. The safest option will be to disallow these
keywords in this context altogether.</p>
Consistency with non-async Rust</h2>
We may want to think of "parallel iteration semantics" as something unique to
async Rust, but in fact non-async Rust faces very similar challenges. To
give a very literal example, the TCP example we used earlier would not be
parallelized in non-async Rust as well:</p>
use </span>std::io;
</span>use </span>std::net::TcpListener;
</span>
</span>let</span> listener = TcpListener::bind("</span>127.0.0.1:8080</span>").await?;
</span>
</span>for</span> stream in listener.</span>incoming</span>() {
</span>    </span>let</span> stream = stream?;
</span>    </span>let </span>(reader, writer) = &</span>mut </span>(&stream, &stream);
</span>    io::copy(reader, writer)?;
</span>}
</span></code></pre>
Networking in non-async Rust is not the best match, but for example a data
processing pipeline would be. And if we forget to parallelize at the right
points there, we'll face very similar issues.</p>
The point being: I don't think "parallelism" is a problem unique to async
Rust. And because of that I don't think we should look for a solution that's
specific to async Rust, but instead look for a solution that can be applied
across domains. I think RFC
2930 (buf_read</code>)</a> is a great example of an
issue that was found in async Rust, but was determined to apply to non-async
Rust as well, and for which a solution was found that can be applied to both
domains.</p>
How would tasks be spawned?</h2>
In order for tasks to be parallelized, they would need to be spawned on a
thread pool somehow. Which means a crucial step in enabling parallel
iteration at the language level is to have a notion of an executor as part of
the language.</p>
C++ is in the process of
adding</a>
to the language (estimate is C++23), so it's not unthinkable for Rust to do
the same. In fact Boats has written about adding a #[global_executor]</code>
before</a>.</p>
// The lowered code from the previous example...
</span>channel.</span>into_par_stream</span>().</span>for_each</span>(async |event| dbg!(event));
</span>
</span>// ... would in turn lower into this
</span>while let </span>Some(event) = channel.</span>next</span>() {
</span>    task::spawn(async </span>move </span>{
</span>        dbg!(event);
</span>    });
</span>}
</span></code></pre>
As you can see we're calling task::spawn</code> for each event. That functionality
would need to come from somewhere, and because async iteration is a language
level concern this would mean that functionality would need to become part of
the language.</p>
I think having a way to perform parallel iteration at the language level is a
great</em> idea. And having a #[global_executor]</code> hook with a default
implementation would solve a lot of issues the ecosystem currently has.</p>
But if history is any indicator this would all take quite a while to shake
out. And I think making async iteration syntax depend on this landing (even
if unstable) would mean it might take a lot longer.</p>
Parallelizing !Send streams</h2>
Not all streams are Send</code>, and async iteration should be able to work with
this as well. We can't quite</em> propagate !Send</code> the way we do for async
blocks; the global executor would need to support spawning !Send</code> futures as
well. This is not a major hurdle as for example async-stc provides
spawn_local</code></a>
function. But it's part of the design space to consider.</p>
Backpressure & concurency control</h2>
For all its warts something that Node.js streams got right was
backpressure</em>. If for example writing to a file is slower than reading from
a file, the pipeline won't keep reading data if the writer isn't available.
This ensures your RAM doesn't fill up with bytes that are waiting to be written.</p>
const </span>fs </span>= </span>require</span>('</span>fs</span>');
</span>
</span>const </span>reader </span>= </span>fs</span>.</span>createReadStream</span>('</span>file.txt</span>');
</span>const </span>compress </span>= </span>zlib</span>.</span>createGzip</span>();
</span>const </span>writer </span>= </span>fs</span>.</span>createWriteStream</span>('</span>file.txt.gz</span>');
</span>
</span>reader</span>.</span>pipe</span>(</span>compress</span>).</span>pipe</span>(</span>writer</span>);
</span></code></pre>
In Rust sequential async iteration applies backpressure by default. A
translation of the above Node.js code would be somewhat like this:</p>
let</span> reader = fs::open("</span>file.txt</span>").await?;
</span>let</span> writer = fs::open("</span>file.txt.gz</span>").await?;
</span>
</span>for</span> buf in GzipEncoder::new(reader) {
</span>    writer.</span>write</span>(buf).await?;
</span>}
</span></code></pre>
async-compression</code>'s Stream</code>
types</a>
already work somewhat like this. Because we're performing sequential async
iteration here, reading data from the stream is constrained by the speed at
which we can write. But also vice-versa: the stream is paused if no new data
is available to be read.</p>
If async iteration syntax was parallel by default we would lose backpressure
by default. Which means if no constraints are specified, by default
parallel async iteration will keep reading data even if there is no consumer
ready to process the data</strong>. Instead limits will always need to be manually
set:</p>
for</span> buf in GzipEncoder::new(reader).</span>limit</span>(</span>5</span>) {
</span>    writer.</span>write</span>(buf).await?;
</span>}
</span></code></pre>
There is a third solution to backpressure we haven't discussed yet: set
some</em> limit by default. Instead of async iteration having unbounded
parallelism by default, we could say, enable 1000</code> items by default. This
limit will however almost always be wrong, and really should be set with
knowledge of the workload.</p>
name</th> max throughput</th> backpressure?</th></tr></thead>

sequential</td> 1</td> yes</td></tr>
bounded</td> n</td> yes</td></tr>
unbounded</td> unlimited</td> no</td></tr>
</tbody></table>
How common is parallel iteration?</h2>
A large part of the argument in favor of parallel semantics by default for
async iteration is that it is the most commonly desired semantics. Perhaps
that may be true today within certain domains; but how true do we expect that
to be in the future for across all domains async Rust will cover?</p>
If we want to get a sense of where async Rust might be headed we can take a
look at Node.js streams: they've been around for a decade longer than Rust
streams have. A common kind of stream in the Node.js ecosystem is the
transform
stream</a>: a
stream that takes an input stream and produces an output stream. Take for
example the csv-parser package</a>: it
takes a stream of CSV text, and produces an output stream of parsed elements:</p>
NAME</span>,</span>AGE
</span>Daffy Duck</span>,</span>24
</span>Bugs Bunny</span>,</span>22
</span></code></pre>
const </span>csv </span>= </span>require</span>('</span>csv-parser</span>')
</span>const </span>fs </span>= </span>require</span>('</span>fs</span>')
</span>const </span>results </span>= [];
</span>
</span>fs</span>.</span>createReadStream</span>('</span>data.csv</span>')
</span>  .</span>pipe</span>(</span>csv</span>())
</span>  .</span>on</span>('</span>data</span>', (</span>data</span>) </span>=> </span>results</span>.</span>push</span>(</span>data</span>))
</span>  .</span>on</span>('</span>end</span>', () </span>=> </span>{
</span>    </span>console</span>.</span>log</span>(</span>results</span>);
</span>    </span>// [
</span>    </span>//   { NAME: 'Daffy Duck', AGE: '24' },
</span>    </span>//   { NAME: 'Bugs Bunny', AGE: '22' }
</span>    </span>// ]
</span>  });
</span></code></pre>
Translating this to async Rust (with async iteration syntax) we could have
something like this:</p>
use </span>async_std::fs::File;
</span>use </span>csv_parser::Csv;
</span>use </span>std::io;
</span>
</span>async </span>fn </span>main</span>() -> io::Result<()> {
</span>    </span>let</span> file = File::open("</span>data.csv</span>").await?
</span>    </span>for</span> item.await? in Csv::new(file) {
</span>        println!("</span>name: </span>{}</span>, age: </span>{}</span>", item["</span>NAME</span>"], item["</span>AGE</span>"]);
</span>    }
</span>}
</span></code></pre>
In this case throughput is likely going to be limited by how fast we can read
from disk, which means spawning tasks for the inner logic is likely not going
to speed this up. In fact, spawning a task per item may in fact slow down
the parser since the logic of "read the file" and "operate on the file" can
no longer be compiled down into a single state machine</strong> 1</a></sup>. While
tasks may be cheap compared to threads, they're by no means without a cost.</p>
^{1</sup>
Inlining performance really depends on the workload. But as a rule
of thumb: if something can't be inlined we're likely going to miss out on
optimizations. Obviously "println" isn't a good example for something that
can be inlined, but you get the gist.</p>
</div>
The example above could be parallelized, if say, we were processing a
directory of files. We could then spawn a task per file. But each file itself
would still be processed as a sequential stream.</p>
Another example of sequential stream semantics is the HTTP/1.1 protocol: the
stream of incoming TCP connections is parallelized, but converting the TCP
stream into a sequence of HTTP requests is sequential. And within the request
itself, each HTTP body is sequential as well. Even a multiplexing protocol
such as HTTP/2 eventually breaks down into request / response pairs which
have sequential semantics:</p>}async </span>fn </span>main</span>() -> io::Result<()> {
</span>    </span>// parallel
</span>    </span>for</span> stream.await? in TcpListener::accept("</span>localhost:8080</span>") {
</span>        </span>// parallel
</span>        </span>for</span> req.await? in Http2Listener::connect(stream) {
</span>            </span>// sequential
</span>            </span>for </span>(i, line?).await in req.</span>lines</span>() {
</span>                println!("</span>line: </span>{}</span>, value: </span>{}</span>", i, line);
</span>            }
</span>        }
</span>    }
</span>}
</span></code></pre>
While parallel iteration may be common, it doesn't seem likely it will ever
be used in such an overwhelming amount that sequential iteration is made
redundant. It seems most likely people will want to mix both in projects, and
we should seek to make both parallel and async iteration convenient.</p>
Compiler hints</h2>
So far much of this post has been about why "parallel semantics for async
iteration" are not the right default. But this doesn't mean that the issue
that intends to address isn't worth addressing. Defaulting to parallel
semantics is only one tool.</p>
What if we could use different tools? For example, one issue raised is:
"people might forget to parallelize certain streams, missing out on
performance". Could we solve this issue through a lint instead? I think we
could, for the most part!</p>
For example clippy could detect known instances of core APIs that are should
probably be parallelized, and warn when they're not. For example handling TCP
connections, or processing files in a directory are things you probably want
to do in parallel.</p>
Or if we wanted to allow anyone to add it to their APIs: Rust could introduce
a #[must_par_spawn]</code> hint that could be added to structs, much like #[must_use]</code>.
It could come with a counterpart tag that marks a function as "this spawns
tasks". And the compiler could warn if something wasn't spawned 2</a></sup>.</p>
^2</sup>There is an RFC being written for the
must_not_await</code></a>
hint; going in this direction would not be without precedent.</p>
</div>
#[</span>must_par_spawn</span>]
</span>struct </span>HttpStream;
</span>
</span>#[</span>par_spawn</span>]
</span>async </span>fn </span>spawn</span><T, F: Future<Output = T> + </span>'static</span>>(</span>f</span>: F) -> T;
</span></code></pre>
Both solutions would need work before implementing; but they present
alternate directions that can be explored instead of resorting to parallel
async iteration syntax.</p>
tool-assisted optimization</h2>
In the post Futures and Segmented
Stacks</a> Boats talks
about techniques to make futures more efficient by moving their stack onto
the heap. Locating where to segment your stack is hard to do by hand, but
could likely become trivial with the right tooling.</p>
Parallelism has many similarities: finding the right places to parallelize
can be tricky. Having a tool that can locate where tasks should be spawned,
and which tasks aren't needed would be great. Runtime profiling for async
programs seems like a generally underexplored domain, and I suspect more and
better tooling could get us really far.</p>
Parallel iteration syntax</h2>
So far in this post we've talked about why async iteration shouldn't default
to parallel semantics by default</em>. But I do agree that it would be great if
we had some</em> syntax for parallel iteration. In the "future directions"
section of my
parallel-stream</a>
post, I laid out what that could look like (using par for</code>):</p>
let mut</span> listener = TcpListener::bind("</span>127.0.0.1:8080</span>").await?;
</span>par </span>for</span> stream.await? in listener.</span>incoming</span>() {
</span>    io::copy(&stream, &stream).await?;
</span>}
</span></code></pre>
In par for</code> loops keywords such as break</code> and continue</code> would be invalid,
and each task would be spawned on a task. This syntax could be made to work
for both sync and non-async Rust, backed by different execution strategies.
And finally, being able to talk about parallelism as a first-class language
concept would enable much better diagnostics, documentation, and tooling to
be implemented.</p>
For example, accidentally writing for x.await</code> instead of par for x.await</code>
on a struct tagged as #[must_par_spawn]</code> could be met with the following
diagnostic:</p>
warning: loop is sequential
</span> --> src/main.rs:2:9
</span>  |
</span>2 |     for stream.await? in listener.incoming() {
</span>  |     ^ help: use `par for stream.await? in listener.incoming()` to parallelize
</span>  |
</span>  = note: `#[warn(sequential_loops)]` on by default
</span></code></pre>
I think parallel iteration syntax in some form would be fantastic</em>. I just
would like to see it exist in a similar way for both async and non-async
Rust.</p>
Addendum: lending streams</h2>
Scottmcm</a> pointed out on Zulip that parallel
iteration semantics also wouldn't work well with
lending streams</a>
(also known as "streaming" or "attached" streams).</p>
In a lending stream the Item</code> that gets returned by Stream</code> may be borrowed
from self. It can only be used as long as the self reference remains live.
For example a Stream</code> where:</p>
type </span>Item<</span>'iter</span>> = &</span>'iter mut</span> State;
</span></code></pre>
...would fail to compile, which seems rather limiting.</p>
Conclusion</h2>
Parallel iteration semantics for Stream</code> are an incredible concept that has
with much potential, but it seems unlikely to be the right fit for all cases.</p>
By defaulting to sequential iteration semantics for Stream</code> parity with
Iterator</code> is preserved. This not only ensures keywords such as break</code> and
continue</code> work in a comparable way, it also enables looking at the design of
"parallel iteration" as a design that can be shared between async and
non-async Rust.</p>
We still have a fair bit to go before we can add async iteration syntax to
Rust. The Stream</code> RFC draft still needs to be submitted and accepted. There
are still questions about migrations and interop with the stream adapters in
the ecosystem. And a design that covers everything from IntoStream</code> to
syntax 3</a></sup>. But even though all that may still be a while; hopefully
this post can serve to anchor discussion on whether iterating over a Stream</code>
should default to sequential or parallel 4</a></sup>.</p>
^{3</sup>
All async iteration syntax in this post has been made up, and is
by no means an indication of what the lang team will decide on. I just needed
something up for the purpose of this post.</p>
</div>}^{4</sup>
I really tried making a joke about "If a stream would wear
iteration, would it wear it this way, or that way" but couldn't get it to
work. My biggest regret in publishing this post.</p>
</div>

Thanks to Friedel Ziegelmayer and Florian Gilcher for proof reading.</em></p>}
Notebooks
Mon, 27 Jul 2020 00:00:00 +0000
Ever since I picked up programming in 2013 I haven't really stopped.
Weekdays, weekends, evenings. I've spent much of my adult life obsessing
about programs. But that's come at a cost: over the years I've started to
realize what I've been trading for the time spent working on projects. And I
realized I needed to change course.</p>
Desiring change is a useful starting point; but it needs to be followed up by
execution. My default mode of doing things was: "drop everything, focus on
the new thing", but that was the root of the problem I was trying to address
1</a></sup>. I realized this wasn't enough, and what I needed was the ability
to formulate plans and the ability to review.</p>
The answer for me has been to adopt a framework: a notebook, pen, and ruler.
In this post I want to share more on how that's worked for me.</p>
^1</sup>I made a list of things I wanted to
do</a> in Jan 2018. That year I only managed to
do the first thing on the list. My 2018 GitHub
contribs</a>
are a testament to the absurd amount of time spent. Looking back at this list
two years later was confrontational and made me realize that I needed to
actively change my approach if I ever wanted to make progress on the things I
cared about.</p>
</div>
bullet journal: a paper OS</h2>
In January I first learned about "bullet journalling" through the Pick Up
Limes channel on YouTube</a>. The
whole video is worth a watch, but this passage really stood out to me:</p>

I genuinely didn't know how I functioned before this system. Because I was
someone who had notes everywhere. [...] It always felt like there was
information coming at me from every direction, but at the same time I
couldn't really find anything when I was looking for it. But now, with this
system, I feel like I've got a little built-in secretary always reminding me
of what I need to do.</p>
</blockquote>
Someone was telling me that a notebook</em> could act as a secratary? That
sounded amazing, and I wanted to learn more! 2</a></sup> So I read the book
"The Bullet Journal Method"</em>. In it the author Ryder Carroll lays out a
framework for taking notes he describes as an "OS for managing your life".</p>
The core tennets of the system are:</p>

A system to quickly take notes and categorize them ("rapid logging").</li>
A time structure to organize lists by (daily, monthly, yearly).</li>
Key moments when and how to interact with the journal.</li>
</ul>
A key reason why this has to be on paper is that the expression of these
ideas is inherently personalized. Every person has different requirements for
what they need to track, and a physical notebook is about the only medium
that can provide this degree of flexibility. It's a system that serves the
needs you have here and now, and adapts to your needs over time.</p>
Another quality of paper is that every action on it takes effort. You cannot
automate a notebook; the most convenient you can make it is to create a
framework. This ensures you have an active relationship with your notes.
Nothing will automatically be added. Nothing will automatically be carried
over. Every entry is deliberate. And this provides opportunity to reflect and
prioritize.</p>
^{2</sup>
Admittedly I had to overcome a kneejerk reaction to dismiss paper
as the right interface to take notes. I suspect this may be due to some
ingrained tech exceptionalism, but I went into it thinking: "How can I turn
this into an app later?" The answer is: it wouldn't work well. Paper is
infinitely versatile and allows adapting layouts to be best suited for you,
at a given time. It's an interface that evolves with you. And no app can
replace that.</p>
</div>}Tools</h2>
It's possible to bullet journal as long as you have a notebook and pen. But
these are the tools I use:</p>

Notebook: LEUCHTTURM1917 A5 hardcover dotted</a> — 18 EUR</li>
Pen: Stabilo point 88, fine 0.4mm</a> — 2 EUR</li>
Ruler: Rumold transparent</a> — 1 EUR</li>
</ul>
I estimate I go through about 3 notebooks per year, estimating my equipment
cost to be about 6 euros per month. I also highly recommend using a
transparent ruler if you can find one, as I found it easier to use than
opaque variants.</p>
Journaling in practice</h2>
Did I say journals are customized to the owner's needs? There is no right</em>
way to journal; but here's how I do it.</p>
The blank spread</h2>

Each page in the journal is empty when you start out. I was so daunted by it
the first time I started: I was afraid of making mistakes and doing things
wrong. A blank page has no system yet; no relationship to any other page
either. And that's okay. We get to define that, and the most important part
is to start.</p>
Daily log</h2>

Every Sunday I plan my week ahead. I create a new daily log spread, and I
copy over all dates and tasks for the week over from my monthly log. And I
copy over the tasks from the week before that I need to do this week. This
means when Monday rolls around I don't start at a deficit, but instead
already know what I need to work on.</p>
My daily journal is a two-page spread that starts on Monday and ends on
Sunday. This leaves 1/8th of the pages over to write notes, which I usually
use to note extra tasks for the week, or tasks to carry over to the next
week.</p>
I start each day with a list of things I need to do (including meetings and
things to read). And end the day with a list of things I have done. If I've
missed something I carry it over to another day, or file it in my monthly
log.</p>
Monthly log</h2>

The monthly log provides a wider breakdown of the month. These are the 14
pages after the yearly log. That is 6 months + 2 pages to capture events
after that. The idea is that we'll run out of notebook pages before we reach
the end of the 6-month period.</p>
On the left hand of each page there's a line per day, with little markings to
delimate each week. And on the right side there's space for todos for the
month. I like to plan this spread out at the start of the month. I filter and
carry over uncompleted tasks from the prior month. And copy over entries from
the yearly log.</p>
The monthly log is also a good place for habit tracking. On the right side of
the left page I have a column marked "e" (for "exercise") that I add a small
x on the day of whenever I've done exercise. This allows me to see at a
glance how I've been doing (17 entries so far this month).</p>
I've recently started experimenting with task prioritization on the right
page. On the bottom half of the page I create two columns: one with things
(projects?) I'm currently working on. And one with goals (projects?) I would
like to be doing. Assuming I need to drop something in order to make time for
something new, I can then figure out how to break it down further.</p>
Yearly log</h2>

The first 4 pages in any notebook make up the yearly log. This is an overview
of each month; with 3 months per page. Here I write down important events,
anniversaries, birthdays, and major upcoming events. I fill this out whenver
I get a new notebook, filtering and carrying over the yearly log from an
earlier notebook. It also serves to plan year-scale events such as holidays
on.</p>
When first starting a notebook this is a fun section to start with. Look up
national holidays. Ask friends when their birthdays are. Schedule and spread
out vacations. Look up conference days, book releases, movie releases.</p>
I usually only consult the yearly log at the start of the month to carry over
events to the monthly log, and whenever I find out about a friend's birthday
3</a></sup>. But hopefully I'll find some other uses for the yearly log in
the future — this is a useful timescale to look at things through, and one I
don't nearly do enough.</p>
^{3</sup>
For any pal reading this who's casually mentioned their
birthday. I've been keeping track, hehe.</p>
</div>
Custom: Daily exercise log</h2>

The spreads I've covered so far are fairly standard. They're almost 1:1
implementations of Ryder Carroll's base templates. I like them because
they're easy to create and take minimal upkeep.</p>
But one project that didn't quite fit in that mold was tracking and planning
exercise. Or well; the way I solve it was by creating another weekly log that
I use to plan and track exercise in.</p>

At the front of my notebook I reserve two pages for for exercises. The left
page contains a collection of sets I can do. And the right page individual
exercises catagorized per muscle group. This much resembles the
meals/groceries spread, but for exercise.</p>
Custom: daily meals/groceries</h2>

I realized last year that my partner and I were ordering in food several
times a week. The reason for that was because we couldn't find the time to
cook, even for fun on the weekends. So I wanted to find a way to lower the
barrier.</p>
The answer ended up being a combination of several things: decide what to
cook ahead of time, make sure we're stocked, and keep a clean cooking surface
at all times. This is still a work in progress, but being able to plan meals
and groceries is a useful tool to have.</p>
The way we do this is by creating a spread with on the left-hand side an
overview of meals for the week: per-day breakdown at the top, and individual
recipes at the bottom. And an overview of ingredients on the right-hand side.</p>
On a good week I sit down with my partner and talk about what we want to cook
that week. Then I do a full index of what we need to prepare those meals
with. I then cross out what we already have in the house from the list. And
finally take a look at the list of staples in the front of the notebook and
check whether we're missing any. We then end up with a decent list of things
we need to buy for the week, and are ready to do a grocery run.</p>
Sometimes we still unexpectedly make plans with pals, or have meetings run
over that prevent cooking meals. But if planned well you roll over meals, or
postpone them for the week after. This takes some practice though.</p>

Figuring out what to buy from the store can be a tedious process. So I like
to keep a list of staples in the front of the notebook (somewhere after the
monthly logs, before the first daily log). It's a useful reference that
allows me to quickly check whether I'm missing something, or I can check it
for inspiration on what to cook.</p>}Custom: sleep log</h2>

One thing I started tracking earlier this year was my sleeping patterns. I
noticed I had trouble falling asleep, trouble waking up, and felt pretty sad
a lot of the time. I figured this would be helpful to track. I no longer
track it (yay good sleep!), but the image above is my sleep for the month
February. 4</a></sup></p>
^{4</sup>
Probably the worst day that month was when I hit 4 and then 5 hours
of sleep, then went for dinner with friends, and had to organize 20 people to
get to a bar. I also caught wind the contract I was negotiating unexpectedly
fell through. The whole week was pretty rough. But it probably would've been
worse if I hadn't been aware of my sleeping patterns.</p>
</div>}Social</h2>
Something new I've started, and am still working on, is keeping track of
interactions with friends and acquiantances. There are people who I used to
interact with regularly but for whatever reason don't interact with enough.
And other people who I wish I was closer with, but am not.</p>
This requires active planning, tracking, and considering. I know how nice it
is when a friend I haven't heard of in a while reaches out — it doesn't have
to be much really. I want to be that person for others. The friend who gets
in touch; who reaches out.</p>
I'm still experimenting with the right system for this, and reading up on
material how to be better (I just started "we should get
together"</a>). Perhaps I'll figure something
out here and have something to share in the future.</p>
Tidying</h2>
Something I've desired to do for a few years has been to be better around the
house. Clean more often, do more chores. Part of this has been communication
with my partner. Part doing research on how to actually keep a house tidy
(konmari actually worked for me). And much of it I've been able to track
through my notebook.</p>
I now clean the windows once every 3 months. I've created a daily habit of
cleaning up in the mornings. I keep a list of chores to do when I have time
available. This has gradually helped me get better around the house, which is
not only nice for the people I live with. It makes me feel more calm and in
touch with where I spend most of my time.</p>
Events</h2>
In the two months before the pandemic my partner and I started going to shows
more. Every month I'd look up events around Berlin, and we'd plan them in to
go see throughout the month.</p>
Because it's so easy to go through the same loop every day. Days turn into
weeks. Weeks into months. And in my case: it feels like little has changed
for me in the last 3 years. Making an active effort to do new things is
something I wanted to do.</p>
Unfortunately the pandemic has thrown a bit of a wrench into this, but I
still wanted to share this actually worked really well for as long as I did
it. With lockdown measures decreasing in Berlin I probably should cautiously
try something like this again: even if it's just parks to visit, areas to go,
or recipes to try.</p>
Conclusion</h2>
This has been an overview of how I use my notebook to track much of my life.
I started doing this 8 months ago, and I feel it's helped me gain more
control of my life. Perhaps you'll find this useful too.</p>
Further Resources</h2>

Bullet Journal on YouTube</a></li>
The Bullet Journal Method book</a> (The audiobook is ~4 hours)</li>
</ul>}}

Yosh Wuyts — Blog

a grand vision for rust

Syntactic musings on the fallibility effect

Placing Arguments

placing functions

Tree-Structured Concurrency II: Replacing Background Tasks With Actors

The background task problem</h2> One of the most common questions people ask when learning about structured concurrency is:</p>

Async Traits Can Be Directly Backed By Manual Future Impls

The Waker Allocation Problem

Open and Closed Effect Systems

Automatic interleaving of high-level concurrent operations

Syntactic musings on match expressions

Syntactic Musings on View Types

A survey of every iterator variant

Musings on iterator trait names

The gen auto-trait problem

What are temporal and spatial memory safety?

Why `Pin` is a part of trait signatures (and why that's a problem)

Solving my RSI issues

Further simplifying self-referential types for Rust

Ergonomic Self-Referential Types for Rust

Self-referential lifetimes</h2> In our motivating example we showed the GivePatsFuture</code> which has the name</code> field that points at the data</code> field. It's a clearly a reference, but it does not carry any lifetime:</p>

in-place construction seems surprisingly simple?

Context Managers: Undroppable Types for Free

Tasks are the wrong abstraction

Shipping Jco 1.0, WASI 0.2

Designing an Async Runtime for WASI 0.2