Async Iteration II: The Async Iterator Crate
— 2022-12-19

1. async iterator
2. future directions

---

1. Async Iteration I: Semantics
2. Async Iteration II: Async Iterator Crate (this post)
3. Async Iteration III: Async Iterator Trait

---

Hey all, today was my first day back from PTO. While away, I was thinking what I often think: I should write more on here. With Twitter effectively defunct, and my Mastodon being set to ephemeral mode (posts auto-delete after 30 days), I should start keeping a better record of what I'm up to somewhere more permanent. And this blog seems as a good a place as any.

Async Iterator

I wanted to share that I've published a crate: async-iterator. Its purpose was partly to test out the new "async fn in traits" feature on nightly. And partly to have a workable sketch of what we expect "async iterator" to behave like.

use async_iterator::prelude::*;

async fn foo(iter: impl Iterator<Item = u32>) -> Vec<u32> {
    iter.collect().await
}

This is not final shape of the trait though, since we're working on keyword generics and expect to retroactively add the async Iterator functionality to the Iterator trait which then becomes generic over "asyncness". But those should just be the same semantics exposed through a different mechanism. Other than that, the crate is pretty minimal, yet covers pretty much all the common usage patterns:

• The base async Iterator trait implemented through async fn next instead of fn poll_next
• The ability to collect into a vec, using actual async IntoIterator and FromIterator bounds
• The ability to async map over values in the iterator
• The ability to extend vec with an async iterator

The only thing missing is async fn filter, which borrows items and so needs some form of "async closures" to work. But all in all this means it works! - "async fn in traits" is real on nightly, and actually does what we want it to do! I think that's really exciting! I don't think the async-iterator crate should be used in actual projects though. It's an interface which only works on nightly, and interfaces only really work when they're uniformly adopted. And between futures 0.3.0 and the stdlib I don't think we should migrate to any in-between interfaces in the interim. But I do think it will be really useful to write experiments against!

Future Directions

I still have to check in with the "async fns in traits" team to see what they're up to (being gone for a month is a long time). But something I might consider doing in the new year is do a rewrite of the async-h1 crate based on "async fn in traits". The internals are basically a giant poll-based state machine, and we've theorized for a while now that rewriting it in terms of async fns would significantly simplify its internals. It seems like this should finally be possible, but I'll have to check with folks first whether doing this will actually be helpful now as well.