Shipping Jco 1.0, WASI 0.2
— 2024-04-03
On this blog I usually post about technical explorations in programming languages, API design, asynchronous computing, and so on. More recently I've re-started my work on WebAssembly, WASI, and their integration with Rust. I expect that work to eventually become more technical again; but my involvement there for the past year has mostly been focused on process and people. And I wanted to take a brief moment to talk about it, because that's the kind of work that's often easily overlooked. And I'd like to take a little victory lap for a second (as a treat).
I can proudly say: I helped ship WASI 0.2! Not just a part of it, but I helped make sure we worked through the last key blocker. And mostly on time at that 1! And I'm super proud of it! The entire project was an incredible team effort - it's hard to understate how hard everyone involved has worked on this. And around five years in the making too. My involvement with the actual design and implementation was minimal, and I was mostly involved in the second half of that period. But I can say I've helped in the following ways:
When first asked to provide estimates I said something along the lines of: "I expect we'll be able to ship in late November at the soonest, but most likely December. December will be tricky because of holidays, but I don't think we'll slip past January." And Jco ended up being ready in mid-January, with WASI 0.2 shipping on January 25th. I think that counts as mostly on time, right?
- Helped ship Jco 1.0: Jco is a native JavaScript WebAssembly toolchain and runtime built for Wasm Components and WASI 0.2. I wrote a blog post about what it is and how to use it on the Bytecode Alliance blog. I basically stepped into the role of PM on that project, ensuring we had clearly defined acceptance criteria to be able to say: "this thing truthfully implements WASI 0.2". And then ended up helping make sure we ended up meeting those criteria. This was my first time donning the PM hat on someone else's project. But that's what was needed, so I was happy to take that on.
- Helped ship WASI 0.2: WASI 0.2 is a rebase of the WebAssembly System Interface onto the WebAssembly Components specification. You can read the announcement post here. Wasmtime had full support for WASI 0.2, I want to say, around September 2023. But in order to ratify the actual specification it needed at least two conforming implementations. My JS days are long behind me; but Jco was the best candidate to ship a second conforming implementation. So I figured I'd do what needed to be done, and stepped up to help make sure it shipped. And in the same meeting we announced Jco had started meeting the conformance criteria, the WASI subgroup committee voted to ratify the WASI 0.2 specification.
- Made the case for a basic async model: One of the biggest limitations of WASI 0.1 was that it wasn't really suited to write networked applications with. I think it was in 2022 that I started making the case that WASI 0.2 probably should ship with some way to support asynchronous sockets if we wanted people to actually use it for server programming. And we shouldn't wait for a future WASI 0.3 to add that potentially years down the line. And that's what we actually ended up shipping: WASI 0.2 supports async sockets and async HTTP via a poll-based interface, making it possible to write asynchronous networked applications today 2.
- Helped ship two new Rust backends: "new" should be in quotation marks
here - one of the backends is basically a rename. But yeah; I helped ship
support for the renamed
wasm32-wasip1target, and the newly introducedwasm32-wasip2targets. My main contributions here were not so much around the implementation, but mostly around writing change proposals, talking to people to help explain the changes and provide assurances, and navigating various organizational challenges. But it's happened, and bar any unexpected issues they should be on the stable channels in a few weeks time! Expect a blog post about this with details on the Rust internals blog soon. (I'm currently procrastinating finishing that one by writing this instead - sorry everyone.) - Supported people: Usually I wouldn't make mention of this as a noteworthy contribution. But there's talking with people and there's talking with people. And over the past two years I've spent a non-trivial amount of time talking with people, listening to what they had to say, and just trying to be there for them. Wasm and WASI are incredibly ambitious projects. I'd roughly describe the work as: imagine being tasked with building several new compilers, defining multiple new operating systems, creating a mix of novel programming language(s) and a shared ABI. Then take all of that and make sure it performs at near-native speeds, is able to be compiled to by virtually every existing programming language, and is usable by non-experts. Then base the entire thing on a capability-based security model and ensure it is perfectly sandboxed by default. Finally, all of that technical work gets compacted into a pressure cooker together with corporate politics, tightening deadlines, and the pressures of working in open source and open standards. Pandemic. Layoffs. Burnout. I hope it's not hard to see why that ends up being A Lot. If there's any takeaway here: sometimes the best open source contribution anyone can make is just to sit and listen to what people have to say.
WASI 0.3's async model is going to be an enormous step up from the current status quo though - and timing-wise the work on that seems to be developing faster than people anticipated. However back in 2022, I was worried that if WASI 0.2 didn't ship with async support, there would be a chance adoption would be too slow to make it to WASI 0.3. The state now is that key industry use cases are unlocked by WASI 0.2, and WASI 0.3 is mainly about making that work better and faster. And at least to me so far this seems to have been the right approach.
And that's it. Unfortunately no cool type theory bits in this post (I like those a lot; I will talk about type systems for hours if left unchecked) - but I still wanted to share a little bit about my other recent work. I often catch myself in the mindset that if I'm not directly coding or designing features, it doesn't actually count as having worked on something. But that's probably unfair to myself: I ended up spending several hundred hours doing useful work to help ship WASI 0.2. And I believe I've actually managed to move the needle on getting WASI in people's hands sooner. Doing things I wasn't necessarily the most comfortable with, but it ended up working out! And while I'm not usually one to gloat, I felt like maybe, perhaps, I do get to gloat a little about this.
I'd also be remiss if I didn't point out that the folks in both the WASI subgroup committee and in the Bytecode Alliance are such a pleasure to work with. People regularly talk about how smart they are, and the quality of the work they produce - but I'd like to instead emphasize just how kind and caring they are. Talented people, unafraid to dream big, who care about both their work and each other. WASI 0.2 and Wasm Components are incredibly impressive, and over the next 6-12 months we're going start seeing the first results of the impact it will have in the space of applied computing.3
To provide an early taste: this headline will probably read like marketing hypeware to most people; some company you've likely never heard of claiming order-of-magnitude improvements in a mature technology. But the technical work underpinning it is legit. And while I'm not in a position to vet the exact claims made, they don't at all seem out of line with what I've seen WASI being able to do so far. I expect similar headlines are going to keep following now that WASI 0.2 has been ratified and is starting to be deployed to production. Exciting times!