Postfix Spawn
— 2022-03-04

1. structured concurrency
2. dangling by default
3. async let
4. postfix spawn
5. design considerations
6. conclusion

Using a free spawn function to create new tasks has been a staple in async Rust for many years. When we introduced async-std in 2019 we took this further by not only providing a spawn function, but fully emulating the std::thread family of APIs and returning. But just because something has been around for a while doesn't mean it's a good idea. And we know we have issues. In this post I want to cover some of those issues, and share an experiment I've been working on which may help with this.

Structured Concurrency

In my previous post I covered the idea of structured concurrency. Though it's worth expanding on this concept more in a separate post ¹, here's the definition we'll be using for the sake of this post:

1. Child tasks should never outlive their parent tasks (this includes destructors).
2. If we cancel a parent task the child task should be cancelled too.
3. If a child task raises an error, the parent task should be able to catch it.

The benefit of this is that code like this will never have dangling tasks, and will largely run in a predictable manner. This has all sorts of interesting properties, such as graceful shutdown trivially falling out of this model - but as I mentioned, that's something for a different post. The takeaway should be that "structured concurrency" is a desirable property for any concurrent code to have.

Dangling by default

As we mentioned in the same post last time, by default most popular runtimes do not behave in a manner which is compatible with structured concurrency. When a JoinHandle is dropped it isn't joined or cancelled, instead it's detached causing it to dangle.

This behavior matches that of std::thread::JoinHandle as well, but as Matklad remarked years ago that behavior in threads is not without issues. In hindsight "join on drop" would have been a better default, but we can't easily change the behavior of std::thread anymore. What we can do is ensure that the behavior of asynchronous tasks default to the right thing. After all: modeling tasks after threads was a choice, we can just as easily make a different choice. std::task is also the one API we already don't expect to want to overload, which means we're not as restricted in the API space as we are with other async stdlib APIs.

So if threads are not the right type to emulate the behavior of for tasks, what might be? I suspect the answer is Future. The futures API RFC mentions that cancelling futures should be done by dropping them. Which means that if our tasks were to emulate futures, we should have "cancel-on-drop" semantics too - as opposed to "join on drop" semantics.

There already exists a runtime which implements this in the wild: smol, which is powered by the async-task crate. "cancel-on-drop" semantics mean that cancellation of an outer task will automatically propagate to inner tasks as well. Making it structurally concurrent by default.

async let

"What if the behavior of futures and tasks were closer to each other" has been an intrusive thought for me lately. Creating futures in Rust is easy, but spawning tasks is significantly harder. Why is that? It turns that the people working on Swift were pondering something similar not too long ago, and they've implemented a solution: async let. Here's a basic example of async let in action:

// given: 
//   func chopVegetables() async throws -> [Vegetables]
//   func marinateMeat() async -> Meat
//   func preheatOven(temperature: Int) async -> Oven

func makeDinner() async throws -> Meal {
  async let veggies = chopVegetables()
  async let meat = marinateMeat()
  async let oven = preheatOven(temperature: 350)

  let dish = Dish(ingredients: await [try veggies, meat])
  return try await oven.cook(dish, duration: .hours(3))
}

Each instance of async let creates a Swift type equivalent to our Task type in Rust. And awaiting an array of them is akin to our future::join operations. Swift provides a global runtime out of the box, and awaiting one of these tasks schedules it on the runtime, and executes it in parallel. This is pretty neat, and compared to their TaskGroup approach for concurrency, it significantly reduces the amount of code needed for many uses ².

What stands out to me is how easy this makes it to introduce ad-hoc parallelism into a codebase. The difference between parallel code and non-parallel code is the addition of a few async keywords sprinkled in front of existing let statements. The ease of this is really cool, and I think we can learn something from that!

Postfix spawn

Now what if we could do this in Rust? Well, not literally the same, but functionally? What if we could make spawning tasks not much harder than creating futures? That'd be nice right? - and I'm happy to share I've got something working as part of the tasky crate. Here's an example of it in use:

use tasky::prelude::*;

let res = async { "nori is a horse" }.spawn().await;
assert_eq!(res, "nori is a horse");

This will convert an existing future to a task, schedule it on the async-std runtime, and await the result. But that's not all; when we call spawn it doesn't actually create a task, it creates an async builder which can be used to configure the task further before it's started. For example, we can use it to give our task a name before spawning it:

let res = async { "nori is a loaf of bread" }
    .spawn()
    .name("nori's tuna stash".into())
    .await;
assert_eq!(res, "nori is a loaf of bread");

The way the "async builder" pattern works is by using the nightly IntoFuture trait. Just like for item in iter loops call IntoIterator on iter. So will .await call IntoFuture on whatever's being awaited. In this case: the TaskBuilder type. This enables us to create chaining builders that are converted into futures when .awaited.

Now what would Swift's earlier example look like if we translated it? For that we'd need the futures-concurrency library too. Let's take a look:

// given: 
//   async fn chop_vegetables() throws -> io::Result<Vec<Vegetables>>
//   async fn marinate_tofu() -> Tofu
//   async fn preheat_oven(temperature: u16) -> Oven

use tasky::prelude::*;
use futures_concurrency::prelude::*;
use std::time::Duration;
use std::io;

async fn make_dinner() -> io::Result<Meal> {
    let veggies = chop_vegetables().spawn();
    let meat = marinate_tofu().spawn();
    let oven = preheat_oven(350).spawn();

    let dish = Dish(&[veggies, meat].try_join().await?);
    oven.cook(dish, Duration::from_secs(60 * 60 * 3)).await?
}

There are still a few warts; but this gets us real close to having semantics as smooth as Swift's. The most awkward pieces of the example are that we don't have Duration::from_hours, and the array requires a try_join method instead of implementing IntoFuture. But other than that: it's quite close!

Converting the Swift example is nice to show that we can achieve parity with Swift, but it doesn't show how how much nicer postfix spawn is over the existing free functions. For that, let's take a look at what it looks like to spawn a longer chain of futures on a task:

// free-function, inline variant
let body = task::spawn(
    surf::get("https://foo.bar")
        .header(some_header)
        .header(other_header)
        .body_json(data)
        .recv_json::<MyResponse>()
).await;

// free-function, variable variant
let req = surf::get("https://foo.bar")
    .header(some_header)
    .header(other_header)
    .body_json(data)
    .recv_json::<MyResponse>();
let body = task::spawn(req).await;

// postfix method
let body = surf::get("https://foo.bar")
    .header(some_header)
    .header(other_header)
    .body_json(data)
    .recv_json::<MyResponse>()
    .spawn()
    .await;

What we're not showing in this example are the extra imports free-functions will always require. Having a postfix spawn function would always just work as part of the Future trait ³.

Having postfix syntax is not only easier to use, it also composes better with existing async features such as .await and IntoFuture. And removing minor speedbumps for common operations is a part of API design that should not be underestimated.

Design Considerations

The implementation of tasky's postfix Future::spawn methods are not yet as polished as I'd like them to be. I knocked them out in about an hour last Friday, and I'm out with COVID right now so I don't see myself trying to polish them anytime soon. Though I do think it might be useful to note what would be done differently if I had the chance though. So let's take a look at the definition:

mod sealed {
    /// Sealed trait to determine what type of bulider we got.
    pub trait Kind {}
}

/// A local builder.
#[derive(Debug)]
pub struct Local;
impl sealed::Kind for Local {}

/// A nonlocal builder.
#[derive(Debug)]
pub struct NonLocal;
impl sealed::Kind for NonLocal {}

/// Task builder that configures the settings of a new task.
pub struct Builder<Fut: Future, K: sealed::Kind> { ... }
impl<Fut> IntoFuture for Builder<Fut, NonLocal> { ... }
impl<Fut> IntoFuture for Builder<Fut, Local> { ... }

pub trait FutureExt: Future + Sized {
    /// Spawn a task on a thread pool
    fn spawn(self) -> Builder<Self, NonLocal>
    where
        Self: Send,
    {
        Builder {
            kind: PhantomData,
            future: self,
            builder: async_std::task::Builder::new(),
        }
    }

    /// Spawn a task on the same thread.
    fn spawn_local(self) -> Builder<Self, Local> {
        Builder {
            kind: PhantomData,
            future: self,
            builder: async_std::task::Builder::new(),
        }
    }
}

We provide two methods on Future: spawn and spawn_local. One spawns Send futures, the other one !Send futures ⁴. The builder needs to know what "mode" it's in to spawn the right future using IntoFuture, so to do that we thread through a generic param which dictates the spawning mode.

The way we currently do this is not ideal. Enum variants aren't types yet, so the best alternative we have is to emulate this using traits and manually written types. Though I think, ideally, we'd be able to differentiate based on a context so that we can be generic over the backend. We don't have a notion yet of what runtime APIs would look like in the stdlib, so perhaps surfacing this as two different traits would actually work well.

The other thing this API hasn't covered is the spawn_blocking API. This convers synchronous operations to futures, backed by a thread pool. We could have a postfix variant of this as well, but I'm not sure whether it's a good idea. In theory we could have a .spawn method for any type T: !Future using some sort of trait. But I'm not sure how much value that would provide over a free function. Personally I'm a bit wary of spawn_blocking, and think if we we were to expose it from the stdlib, a free function would probably suffice.

Conclusion

async-std first proposed the model of: "What if tasks are async threads". smol moved away from this by providing "cancel-on-drop" semantics. In this post we showed some of the benefits gained by moving further away from that model, and closer to: "What if tasks are parallelizable futures?"

In the future I'd very much like to explore whether we could move even closer to the "tasks-as-futures" model. Right now the tasky crate allows people to break structured concurrency guarantees by calling JoinHandle::detach or mem::forget(handle):

use tasky::prelude::*;

async { loop {} } // runs forever on a thread
    .spawn()
    .into_future()
    .detach();

What if we made it so tasks aren't spawned on executors until .awaited, just like futures? What if we did away with detach entirely? We can't guarantee that all runtimes behave this way, since people can implement their own. But for the stdlib, would that make sense? This would still use "cancel on drop semantics" since .await points mark cancellation sites. But we'd also gain "tasks aren't started until .awaited", just like futures. Just how much do we value having structured concurrency? Just what exactly does eager execution of tasks provide us right now?

Structured concurrency increases safety in concurrent programs, and I think it's exploring in closer detail what it provides, and what we need to do to achieve it. But until then: I hope you enjoyed this post! I wrote this because indexing a bunch of the Swift proposals, learned about async let, and got excited. And I hope you're excited it now too!