hckrnws

As it happens i have an even better API than this article proposes!

They propose just using an async iterator of UInt8Array. I almost like this idea, but it's not quite all the way there.

They propose this:

  type Stream<T> = {
    next(): Promise<{ done, value: UInt8Array<T> }>
  }

  type Stream<T> = {
    next(): { done, value: T } | Promise<{ done, value: T }>
  }

- I can easily make mine from theirs

- In theirs the conceptual "stream" is defined by an iterator of iterators, meaning you need a for loop of for loops to step through it. In mine it's just one iterator and it can be consumed with one for loop.

- I'm not limited to having only streams of integers, they are

- My way, if I define a sync transform over a sync input, the whole iteration can be sync making it possible to get and use the result in sync functions. This is huge as otherwise you have to write all the code twice: once with sync iterator and for loops and once with async iterators and for await loops.

- The problem with thrashing Promises when splitting input up into words goes away. With async iterators, creating two words means creating two promises. With stream iterators if you have the data available there's no need for promises at all, you just yield it.

- Stream iterators can help you manage concurrency, which is a huge thing that async iterators cannot do. Async iterators can't do this because if they see a promise they will always wait for it. That's the same as saying "if there is any concurrency, it will always be eliminated."

> Obviously I'm gonna be biased, but I'm pretty sure my version is also objectively superior:

> - I can easily make mine from theirs

That... doesn't make it superior? On the contrary, theirs can't be easily made out of yours, except by either returning trivial 1-byte chunks, or by arbitrary buffering. So their proposal is a superior primitive.

On the whole, I/O-oriented iterators probably should return chunks of T, otherwise you get buffer bloat for free. The readv/writev were introduced for a reason, you know.

> So their proposal is a superior primitive.

This lines up with my thinking. The proposal should give us a building block in the form of the primitive. I would expect the grandparent comment’s API to be provided in a library built on top of a language level primitive.

How would you then deal with a stream of UTF8 code points? They won't fit in a UInt8Array. There will be too many for async iterators to perform well: you'll hit the promise thrashing issues discussed in the blog post

No, you'll just need to (potentially) keep the last 1-2 bytes of the previous chunk after each iteration. Come on, restartable UTF-8 APIs has been around for more than 30 years.

But those code points were just inputs to another stream transformation that turns a stream of code points into a stream of graphemes. Rapidly your advice turns into "just do everything in one giant transformation" and that loses the benefits of streams, which are meant to be highly composable to create efficient, multi-step transformation pipelines.

What's stopping you from implementing a stream transformation that reads the raw stream like a parser, outputting a grapheme or whatever unit you want only when it knows it's done reading it from the input?

No, it doesn't turn into this. Those two bytes of leftovers plus a flag are kept inside the stream generator that transforms bytes into code points, every time you pull it those two bytes are used as an initial accumulator in the fold that takes the chunk of bytes and yield chunk of code points and the updated accumulator. You don't need to inline it all into one giant transform.

Come on, it's how (mature libraries of) parser combinators work. The only slightly tricky part here is detecting leftover data in the pipeline.

To quote the article:

> If you want to stream arbitrary JavaScript values, use async iterables directly

OK, so we have to do this because code points are numbers larger than 8 bits, so they're arbitrary JS values and we have to use async iterables directly. This is where the amount of per-item overhead in an async iterable starts to strangle you because most of the actual work being done at that point is tearing down the call stack between each step of each iterator and then rebuilding it again so that the debugger has some kind of stack traces (if you're using for await of loops to consume the iterables that is).

As an abstraction I would say it does make mine superior that it captures everything theirs can and more that theirs can't.

Plus theirs involves the very concrete definition of an array, which might have 100 prototype methods in JS, each part of their API surface. I have one function in my API surface.

I think the more generic stream concept is interesting, but their proposal is based on different underlying assumptions.

From what it looks like, they want their streams to be compatible with AsyncIterator so it'd fit into existing ecosystem of iterators.

And I believe the Uint8Array is there for matching OS streams as they tend to move batches of bytes without having knowledge about the data inside. It's probably not intended as an entirely new concept of a stream, but something that C/C++ or other language that can provide functionality for JS, can do underneath.

For example my personal pet project of a graph database written in C has observers/observables that are similar to the AsyncIterator streams (except one observable can be listened to by more than one observer) moving about batches of Uint8Array (or rather uint8_t* buffer with capacity/count), because it's one of the fastest and easiest thing to do in C.

It'd be a lot more work to use anything other than uint8_t* batches for streaming data. What I mean by that, is that any other protocol that is aware of the type information would be built on top of the streams, rather than being part of the stream protocol itself for this reason.

Yeah it makes sense to me that the actual network socket is going to move data around in buffers. I'm just offering an abstraction over that so that you can write code that is wholly agnostic to how data is stored.

And yes, because it's a new abstraction the compat story is interesting. We can easily wrap any source so we'll have loads of working sources. The fight will be getting official data sinks that support a new kind of stream

I did a microbenchmark recently and found that on node 24, awaiting a sync function is about 90 times slower than just calling it. If the function is trivial, which can often be the case.

If you go back a few versions, that number goes up to around 105x. I don’t recall now if I tested back to 14. There was an optimization to async handling in 16 that I recall breaking a few tests that depended on nextTick() behavior that stopped happening, such that the setup and execution steps started firing in the wrong order, due to a mock returning a number instead of a Promise.

I wonder if I still have that code somewhere…

that sounds way off. there is a big perf hit to async, but it appears to be roughly 100 nanoseconds overhead per call. when benchmarking you have to ensure your function is not going to be optimized away if it doesn't do anything or inputs/outputs never change.

you can run this to see the overhead for node.js Bun and Deno: https://gist.github.com/billywhizz/e8275a3a90504b0549de3c075...

> I did a microbenchmark recently and found that on node 24, awaiting a sync function is about 90 times slower than just calling it. If the function is trivial, which can often be the case.

I dabble in JS and… what?! Any idea why?

Any await runs the logic that attempts to release the main message pump to check for other tasks or incoming IO events. And it looks like that takes around 90 instructions to loop back around to running the next line of the code, when the process is running nothing else.

If you’re doing real work, 90 instructions ain’t much but it’s not free either. If you’ve got an async accumulator (eg, otel, Prometheus) that could be a cost you care about.

How did you come up with 90? Can you shed any might on the difference between the cost of promise resolution and the cost of await? Is there any cost component with how deep in the call stack you are when an await happens?

Essentially for loop of 10k iterations comparing `fn()` versus `await fn()` fed into a microbenchmark tool, with some fiddling to detect if elimination was happening or ordering was changing things.

I was bumping into PRs trying to eliminate awaits in long loops and thinking surely the overhead can’t be so high to warrant doing this, especially after node ~16. I was wrong.

Clarifying:

‘fn()’ is the exact same function call in both cases. Only the benchmark itself was async versus sync.

Because the case under test is what’s the cost of making an async api when 90% of the calls are sync? And the answer is a lot higher than I thought.

I assume the reason is that `await` de-schedules the current microtask. In fact, even if you immediately return from an `await`, the de-scheduling can introduce behavior that otherwise would be absent without `await`. For this reason, code optimizers (like the Google Closure Compiler) treat `await` as a side-effect and do not optimize it out.

Here is my test harness and results: https://github.com/conartist6/async-perf

There is no such thing as Uint8Array<T>. Uint8Array is a primitive for a bunch of bytes, because that is what data is in a stream.

Adding types on top of that isn't a protocol concern but an application-level one.

A Uint8Array can be backed by buffers other than ArrayBuffer, which is where the types [0] come from.

[0] https://github.com/microsoft/TypeScript/blob/924810c077dd410...

> Adding types on top of that isn't a protocol concern but an application-level one.

I agree with this.

I have had to handle raw byte streams at lower levels for a lot of use-cases (usually optimization, or when developing libs for special purposes).

It is quite helpful to have the choice of how I handle the raw chunks of data that get queued up and out of the network layer to my application.

Maybe this is because I do everything from C++ to Javascript, but I feel like the abstractions of cleanly getting a stream of byte arrays is already so many steps away from actual network packet retrieval, serializing, and parsing that I am a bit baffled folks want to abstract this concern away even more than we already do.

I get it, we all have our focuses (and they're ever growing in Software these days), but maybe it's okay to still see some of the bits and bytes in our systems?

My concern isn't with how you write your network layer. Use buffers in there, of course.

But what if you just want to do a simple decoding transform to get a stream of Unicode code points from a steam of bytes? If your definition of a stream is that it has UInt8 values, that simply isn't possible. And there's still gonna be waaay too many code points to fall back to an async iterator of code points.

I think we're having a completely different conversation now. The parent comment I originally replied has been edited so much that I think the context of what I was referring to is now gone.

Also, I wasn't talking about building network layers, I was explicitly referring to things that use a network layer... That is, an application receiving streams of enumerable network data.

I also agree with what you're saying, we don't want UInt8, we want bits and bytes.

I'm really confused as to why the parent comment was edited so heavily. Oh well, that's social media for you.

Not the person originally replying, but as someone who avoids JS I have to ask whether the abstraction you provide may have additional baggage as far as framing/etc.

Ironically, naively, I'd expect something more like a callback where you would specify how your input gets written to a buffer, but again im definitely losing a lot of nuance from not doing JS in a long while.

This is similar to how Clojure transducers are implemented: "give me the next thing plz." – https://clojure.org/reference/transducers

Other angles of critique & consideration already covered well by sibling commenters. One extra consideration (unrelated to streams, more general) is the API design & dev UX/DX:

  type Stream<T> = {
    next(): { done, value: T } | Promise<{ done, value: T }>
  }

  type Stream<T> = {
    next(): { done, value: T }
  }
  type Stream<T> = {
    next(): Promise<{ done, value: T }>
  }

> My way, if I define a sync transform over a sync input, the whole iteration can be sync making it possible to get and use the result in sync functions. This is huge as otherwise you have to write all the code twice: once with sync iterator and for loops and once with async iterators and for await loops.

Writing all the code twice is cleaner in every implementation scenario I can envisage. It's very rare I want generalised flexibility on an API call - that leads to a lot of confusion & ambiguity when reading/reviewing code, & also when adding to/editing code. Any repetitiveness in handling both use-cases (separately) can easily be handled with well thought-out composition.

How is it cleaner? I used to actually do that. I wrote everything twice. I even built fancy tools to help me write everything twice.

But the bigger problem here is that sync and async aren't enough. You almost need to write everything three times: sync, async, and async-batched. And that async-batched code is gonna be gnarly and different from the other two copies and writing it in the first place and keeping it in sync is gonna give you headaches.

To see how it played out for me take a look at the difference between:

https://github.com/iter-tools/regex/blob/a35a0259bf288ccece2... https://github.com/iter-tools/regex/blob/a35a0259bf288ccece2... https://github.com/iter-tools/regex/blob/a35a0259bf288ccece2...

I think the context that some other responders are missing is that in some functional languages, like Elixir, streams and iterators are used idiomatically to do staged transforms of data without necessitating accumulation at each step.

They are those languages versions of goroutines, and JavaScript doesn’t have one. Generators sort of, but people don’t use them much, and they don’t compose them with each other.

So if we are going to fix Streams, an implementation that is tuned only for IO-bound workflows at the expense of transform workflows would be a lost opportunity.

Your idea is flatten the UInt8Array into the stream.

While I understand the logic, that's a terrible idea.

* The overhead is massive. Now every 1KiB turns into 1024 objects. And terrible locality.

* Raw byte APIs...network, fs, etc fundamentally operate on byte arrays anyway.

In the most respectful way possible...this idea would only be appealing to someone who's not used to optimizing systems for efficiency.

JS engines actually are optimized to make that usage pattern fast.

Small, short-lived objects with known key ordering (monomorphism) are not a major cost in JS because the GC design is generational. The smallest, youngest generation of objects can be quickly collected with an incremental GC because the perf assumption is that most of the items in the youngest generation will be garbage. This allows collection to be optimized by first finding the live objects in the gen0 pool, copying them out, then throwing away the old gen0 pool memory and replacing it with a new chunk.

What happens when I send an extremely high throughput of data and the scheduler decides to pause garbage collection due to there being too many interrupts to my process sending network events? (a common way network data is handed off to an application in many linux distros)

Are there any concerns that the extra array overhead will make the application even more vulnerable to out of memory errors while it holds off on GC to process the big stream (or multiple streams)?

I am mostly curious, maybe this is not a problem for JS engines, but I have sometimes seen GC get paused on high throughput systems in GoLang, C#, and Java, which causes a lot of headaches.

Yeah I don't think that's generally a problem for JS engines because of the incremental garbage collector.

If you make all your memory usage patterns possible for the incremental collector to collect, you won't experience noticeable hangups because the incremental collector doesn't stop the world. This was already pretty important for JS since full collections would (do) show up as hiccups in the responsiveness of the UI.

Interesting, thanks for the info, I'll do some reading on what you're saying. I agree, you're right about JS having issues with hiccups in the UI due to scheduling on a single process thread.

Makes a lot of sense, cool that the garbage collector can run independently of the call stack and function scheduler.

OP doesn’t know what he’s talking about. Creating an object per byte is insane to do if you care about performance. It’ll be fine if you do 1000 objects once or this isn’t particularly performance sensitive. That’s fine. But the GC running concurrently doesn’t change anything about that, not to mention that he’s wrong and the scavenger phase for the young generation (which is typically where you find byte arrays being processed like this) is stop the world. Certain phases of the old generation collection are concurrent but notably finalization (deleting all the objects) is also stop the world as is compaction (rearranging where the objects live).

This whole approach is going to be orders of magnitude of overhead and the GC can’t do anything because you’d still be allocating the object, setting it up, etc. Your only hope would be the JIT seeing through this kind of insanity and rewriting to elide those objects but that’s not something I’m aware AOT optimizer can do let alone a JIT engine that needs to balance generating code over fully optimal behavior.

Don’t take my word for it - write a simple benchmark to illustrate the problem. You can also look throughout the comment thread that OP is just completely combative with people who clearly know something and point out problems with his reasoning.

Even if you stop the world while you sweep the infant generation, the whole point of the infant generation is that it's tiny. Most of the memory in use is going to be in the other generations and isn't going to be swept at all: the churn will be limited to the infant generation. That's why in real usage the GC overhead is I would say around 15% (and why the collections are spaced regularly and quick enough to not be noticeable).

I've been long on JS but never heard things like this, could you please prove it by any means or at least give a valid proof to the _around 15%_ statement? Also by saying _quick enough to not be noticeable_, what's the situation you are referring too? I thought the GC overhead will stack until it eventually affects the UI responsiveness when handling continues IO or rendering loads, as recently I have done some perf stuff for such cases and optimizing count of objects did make things better and the console definitely showed some GC improvements, you make me nerve to go back and check again.

Yeah I mean don't take my word, play around with it! Here's a simple JSFiddle that makes an iterator of 10,000,000 items, each with a step object that cannot be optimized except through efficient minor GC. Try using your browser's profiler to look at the costs of running it! My profiler says 40% of the time is spent inside `next()` and only 1% of the time is spent on minor GCs. (I used the Firefox profiler. Chrome was being weird and not showing me any data from inside the fiddle iframe).

JSFiddle link missing.

sry https://jsfiddle.net/L10dw6gn/3/

The allocation of each object still has overhead though, even if they all live side-by-side. You get memory overhead for each value. A Uint8Array is tailor-made for an array of bytes and there’s a constant overhead. Plus the garbage collector doesn’t even have to peer inside a Uint8Array instance.

The engine can optimize all those allocations out of existence so they never happen at all, so it's not a problem we'll be stuck with forever, just a temporary inconvenience.

If a generator is yielding values it doesn't expose step objects to its inner code. If a `for of` loop is consuming yielded values from that generator, step objects are not exposed directly to the looping code either.

So now when you have a `for of` loop consuming a generator you have step objects which only the engine ever can see, and so the engine is free to optimize the allocations away.

The simplest way the engine could do it is just to reuse the same step object over and over again, mutating step.value between each invocation of next().

It's not blazingly fast, no, but it's not as much overhead as people think either when they're imagining what it would cost to do the same thing with malloc. TC39 knew all this when they picked { step, done } as the API for iteration and they still picked it, so I'm not really introducing new risk but rather trusting that they knew what they were doing when they designed string iterators.

At the moment the consensus seems to be that these language features haven't been worth investing much in optimizing because they aren't widely used in perf-critical pathways. So there's a chicken and egg problem, but one that gives me some hope that these APIs will actually get faster as their usage becomes more common and important, which it should if we adopt one of these proposed solutions to the current DevX problems

Brother, you are talking about one object for every byte.

That is a madness. And often for no reason...you're copying or arranging bytes in lists anyway.

That's just how string iterators work in Javascript: one object for every byte. For now it's fast enough: https://v8.dev/blog/trash-talk. I'd put the GC overhead at around 10-15%, even with 20+ objects per byte when you add up all the stages in a real text processing pipeline. It's that cheap because the objects all have short lifespans and so they spend all their lives in the "tiny short-lived objects" memory pool which is super easy to incrementally GC.

In the future it should be entirely possible for the engines to optimize even more aggressively too: they should be to skip making the object if the producer of values is a generator function and the consumer is a for loop.

I agree with your post, but in practice, couldn't you get back that efficiency by setting T = UInt8Array? That is, write your stream to send / receive arrays.

My reference point is from a noob experience with Golang - where I was losing a bunch of efficiency to channel overhead from sending millions of small items. Sending batches of ~1000 instead cut that down to a negligible amount. It is a little less ergonomic to work with (adding a nesting level to your loop).

Yes, then you are back to Cloudflare's suggested interface.

An async iterator of buffers.

There's one more interesting consequence: you rid yourself of the feedback problem.

To see the problem let's create a stream with feedback. Lets say we have an assembly line that produces muffins from ingredients, and the recipe says that every third muffin we produce must be mushed up and used as an ingredient for further muffins. This works OK until someone adds a final stage to the assembly line, which puts muffins in boxes of 12. Now the line gets completely stuck! It can't get a muffin to use on the start of the line because it hasn't made a full box of muffins yet, and it can't make a full box of muffins because it's starved for ingredients after 3.

If we're mandated to clump the items together we're implicitly assuming that there's no feedback, yet there's also no reason that feedback shouldn't be a first-class ability of streams.

How do you send multiple sub-streams in parallel?

In the language I've been working on for a couple months, Eidos, streams are achieved through iterators as well. It's dead simple. And lazy for loops are iterators, and there is piping syntax. This means you can do this (REPL code):

  >> fn double(iter: $iterator<i32>) {
    return *for x in iter { $yield( x * 2 )}
  }

  >> fn add_ten(iter: $iterator<i32>) {
    return *for x in iter { $yield( x + 10 )}
  }

  >> fn print_all(iter: $iterator<i32>) {
    for x in iter { $print( x )}
  }

  >> const source = *for x in [1, 2, 3] { $yield( x )}

  >> source |> double |> add_ten |> print_all
  12
  14
  16

Async iterables aren't necessarily a great solution either because of the exact same promise and stack switching overhead - it can be huge compared to sync iterables.

If you're dealing with small objects at the production side, like individual tag names, attributes, bindings, etc. during SSR., the natural thing to do is to just write() each string. But then you see that performance is terrible compared to sync iterables, and you face a choice:

  1. Buffer to produce larger chunks and less stack switching. This is the exact same thing you need to do with Streams. or

  2. Use sync iterables and forgo being able to support async components.

We faced this problem in Lit-SSR and our solution was to move to sync iterables that can contain thunks. If the producer needs to do something async it sends a thunk, and if the consumer receives a thunk it must call and await the thunk before getting the next value. If the consumer doesn't even support async values (like in a sync renderToString() context) then it can throw if it receives one.

This produced a 12-18x speedup in SSR benchmarks over components extracted from a real-world website.

I don't think a Streams API could adopt such a fragile contract (ie, you call next() too soon it will break), but having some kind of way where a consumer can pull as many values as possible in one microtask and then await only if an async value is encountered would be really valuable, IMO. Something like `write()` and `writeAsync()`.

The sad thing here is that generators are really the right shape for a lot of these streaming APIs that work over tree-like data, but generators are far too slow.

Yeah that problem you have is pretty much what I'm offering a solution to. It's the same thing you're already doing but more robust.

Also I'm curious why you say that generators are far too slow. Were you using async generators perhaps? Here's what I cooked up using sync generators: https://github.com/bablr-lang/stream-iterator/blob/trunk/lib...

This is the magic bit:

  return step.value.then((value) => {
    return this.next(value);
  });

You know now that I look at it I do think I need to change this code to defend better against multiple eager calls to `next()` when one of them returns a promise. With async generators there's a queue built in but since I'm using sync generators I need to build that defense myself before this solution is sound in the face of next();next(). That shouldn't be too hard though.

I liked conartist6's proposal,

  type Stream<T> = {
    next(): { done, value: T } | Promise<{ done, value: T }>
  }

Engineers had a collective freak out panic back in 2013 over Do not unleash Zalgo, a worry about using callbacks with different activation patterns. Theres wisdom there, for callbacks especially; it's confusing if sometime the callback fires right away, sometimes is in fact async. https://blog.izs.me/2013/08/designing-apis-for-asynchrony/

And this sort of narrow specific control has been with us since. It's generally not cool to use MaybeAsync<T> = T | Promise<T>, for similar "it's better to be uniform" reasons. We've been so afraid of Zalgo for so long now.

That fear just seems so overblown and it feels like it hurts us so much that we can't do nice fast things. And go async when we need to.

Regarding the pulling multiple, it really depends doesn't it? It wouldn't be hard to make a utility function that lets you pull as many as you want queueing deferrables, allowing one at a time to flow. But I suspect at least some stream sources would be just fine yielding multiple results without waiting. They can internally wait for the previous promise, use that as a cursor.

I wasn't aware that generators were far too slow. It feels like we are using the main bit of the generator interface here, which is good enough.

Yeah I think people took away "It's better to be uniform" since they were trying to block out the memory of much-feared Zalgo, but if you read the article carefully it says in big letters "Avoid Synthetic Deferrals" then goes on to advocate for patterns exactly like MaybeAsync to be used "if the result is usually available right now, and performance matters a lot".

I was so sick of being slapped around by LJHarb who claimed to me again and again that TC39 was honoring the Zalgo post (by slapping synthetic deferrals on everything) that I actually got Isaacs to join the forum and set him straight: https://es.discourse.group/t/for-await-of/2452/5

That's an amazing thread, thanks for posting it! I've wanted `for await?()` for exactly these situations.

I feel like my deep dives into iterator performance are somewhat wasted because I might have made my project faster, but it's borderline dark magic and doesn't scale to the rest of the ecosystem because the language is broken.

The practical pain with Web Streams in Node.js is that they feel like they were designed for the browser use case first and backported to the server. Any time I need to process large files or pipe data between services, I end up fighting with the API instead of just getting work done.

The async iterable approach makes so much more sense because it composes naturally with for-await-of and plays well with the rest of the async/await ecosystem. The current Web Streams API has this weird impedance mismatch where you end up wrapping everything in transform streams just to apply a simple operation.

Node's original stream implementation had problems too, but at least `.pipe()` was intuitive. You could chain operations and reason about backpressure without reading a spec. The Web Streams spec feels like it was written by the kind of person who thinks the solution to a complex problem is always more abstraction.

It's news to me that anyone actually uses the web streams in node. I thought they were just for interoperability, for code that needs to run on both client and server.

You need to use them for things like Cloudflare and Denos HTTP servers, which is actually a fairly common (and nice) pattern:

https://blog.val.town/blog/the-api-we-forgot-to-name/

> Two years ago Cloudflare released an API for creating servers in JavaScript. Now every modern JavaScript cloud provider supports it.

This is so ridiculously far from the truth lol. Every JS runtime after node has been championing web APIs and that’s how you get the fetch API’s Request/Response outside the browser.

A long time ago, I wrote an abstraction called a Repeater. Essentially, the idea behind it is, what would the Promise constructor look like if it was translated to async iterables.

  import { Repeater } from "@repeaterjs/repeater";
  
  const keys = new Repeater(async (push, stop) => {
    const listener = (ev) => {
      if (ev.key === "Escape") {
        stop();
      } else {
        push(ev.key);
      }
    };
    window.addEventListener("keyup", listener);
    await stop;
    window.removeEventListener("keyup", listener);
  });
  const konami = ["ArrowUp", "ArrowUp", "ArrowDown", "ArrowDown", "ArrowLeft", "ArrowRight", "ArrowLeft", "ArrowRight", "b", "a"];
  (async function() {
    let i = 0;
    for await (const key of keys) {
      if (key === konami[i]) {
        i++;
      } else {
        i = 0;
      }
      if (i >= konami.length) {
        console.log("KONAMI!!!");
        break; // removes the keyup listener
      }
    }
  })();

It’s one of those abstractions that’s feature complete and stable, and looking at NPM it’s apparently getting 6.5mil+ downloads a week for some reason.

Lately I’ve just taken the opposite view of the author, which is that we should just use streams, especially with how embedded they are in the `fetch` proposals and whatever. But the tee critique is devastating, so maybe the author is right. It’s exciting to see people are still thinking about this. I do think async iterables as the default abstraction is the way to go.

In the repeater callback, you're both calling the stop argument and awaiting it. Is it somehow both a function and a promise? Is this possible in JS?

edit: I found where stop is created[1]. I can't say I've seen this pattern before, and the traditionalist in me wants to dislike the API for contradicting conventions, but I'm wondering if this was designed carefully for ergonomic benefits that outweigh the cost of violating conventions. Or if this was just toy code to try out new patterns, which is totally legit also

[1]: https://github.com/repeaterjs/repeater/blob/638a53f2729f5197...

It's not common, but there are web APIs where you await a promise that already exists as a property, rather than being returned from a function call, like

    await document.fonts.ready

    device.lost.then(() => {
      console.log('WebGPU device lost :(')
    })

Yes, the callable promise abstraction is just a bit of effort:

  let resolveRef;
  const promise = new Promise((res) => { resolveRef = res; });
  
  const callback = (data) => {
    // Do work...
    resolveRef(data); // This "triggers" the await
  };

  Object.assign(callback, promise);

Off topic - But just wanna say - Love the cheat code! 30 Lives added :-) Nostalgia runs deep with that code. So deep - in fact, that I sign many of my emails off with "Sent by hitting Up, Up, Down, Down, Left, Right, Left, Right, B, A"

Off topic to the off topic, but that logic doesn't look right. It seems like if up is pressed, you might need to reset i to 1 or 2, not 0.

Not accepting PRs for this. I think the logic is sound (Easter Eggs should be difficult to trigger).

The tension between "streams as lazy sequences" vs "streams as async event channels" isn't unique to JavaScript. Every major runtime has hit this wall:

- Java went through it with java.util.stream (pull-based, lazy) vs Reactive Streams/Project Reactor (push-based, backpressure-aware). The result was two completely separate APIs that don't compose well.

- .NET actually handled this better with IAsyncEnumerable<T> in C# 8 — a single abstraction that's pull-based but async-aware. It composes naturally with LINQ and doesn't require a separate reactive library for most use cases.

- Go side-stepped the problem entirely with goroutines and channels, making the whole streams abstraction unnecessary for most cases.

What I find interesting about this proposal is it's trying to learn from that prior art. The biggest mistake Java made was bolting async streams on top of a synchronous abstraction and then needing a completely separate spec (Reactive Streams) for the async case. If JavaScript can get a single unified abstraction that handles both sync iteration and async backpressure, that would be a genuine improvement over what exists in most other runtimes.

> This pattern has caused connection pool exhaustion in Node.js applications using undici (the fetch() implementation built into Node.js), and similar issues have appeared in other runtimes.

That's an inherent flaw of garbage collected languages. Requiring to explicitly close a resource feels like writing C. Otherwise you have a memory leak or resource exhaustion, because the garbage collector may or may not free the resource. Even C++ is better at this, because it does reference counting instead.

One minor niggle on freeing resources... I'm hoping it becomes more popular with libraries, but there's using/await using with disppse/disposeAsync which works similarly to C#'s use of using.

I'm working on a db driver that uses it by convention as part of connection/pool usage cleanup.

Seems pretty similar to the design of OKIO in java [1]. With pretty similar goals ultimately. Here's a presentation on the internal details and design decisions. [2]

[1] https://github.com/square/okio

[2] https://www.youtube.com/watch?v=Du7YXPAV1M8

I don’t know how ReadableStream.tee() got specified to backpressure when the faster branch is not consumed, since this is the opposite of what nodejs does when multiple Writables attached via Readable.pipe() and also the opposite of what the requirements document (https://github.com/whatwg/streams/blob/e9355ce79925947e8eb49...) says: “letting the speed of the slowest output determine the speed of the tee”.

I like the idea of the more ergonomic, faster api in new-stream with no buffering except at Stream.push(). NodeJS and web streams put infinitely expandable queues at every ReadableStream and WritableStream so that you can synchronously res.write(chunk) as much as you want with abandon. This API basically forces you to use generators that yield instead of synchronously writing chunks.

Regarding the benchmarks, "Async iteration (8KB × 1000): ~530 GB/s vs ~35 GB/s": how do you achieve 530 GB/s throughput on an M1 Pro which has a 200GB/s memory bandwidth? The "~275 GB/s" figure for chained transforms has the same problem.

I suspect the benchmarks, if not most of this project, suffer from poor quality control on vibecoded implementations.

Web Streams do feel rather painful compared to other languages. The author ends up basically describing kotlin flows which are great and I wish the web would adopt that model (Observables wanted to be that but the API is much worse than flows in practice).

Fwiw the original Streams API could have been simpler even without async iterators.

  interface Stream<T> {
    // Return false from the callback to stop early.
    // Result is if the stream was completed.
    forEach(callback: (chunk: T) => Promise<boolean | undefined>): Promise<boolean>
  }

If we're optimizing allocations we can also avoid all the {done,value} records and return a semaphore value for the end in the proposed API.

(Web) API design is really difficult and without a voice in the room pushing really hard on ergonomics and simplicity it's easy to solve all the use cases but end up with lots of awkward corners and costs later.

As I’m sure many have, I wrote a wrapper around AyncIterables so I could use them more succinctly. However I wasn’t concerned with performance as I was using it in a lambda handling small batches or scanning items from a DB and streaming pages back.

https://github.com/juliantcook/fluent-async-iterator

I had hoped we would have a better API by now.

It was also very useful for CLI tools utilising unix pipes.

I ran into a performance issues a few months ago where native streams were behaving terribly, and it seemed to be due to bad back-pressure implementation.

I tried several implementations, tweaked settings, but ultimately couldn't get around it. In some cases I had bizarre drops in activity when the consumer was below capacity.

It could have been related to the other issue they mention, which is the cost of using promises. My streams were initiating HEAPS of promises. The cost is immense when you're operating on a ton of data.

Eventually I had to implement some complex logic to accomplish batching to reduce the number of promises, then figure out some clever concurrency strategies to manage backpressure more manually. It worked well.

Once I was happy with what I had, I ported it from Deno to Go and the result was so stunningly different. The performance improvement was several orders of magnitude.

I also built my custom/native solution using the Effect library, and although some people claim it's inefficient and slow, it out-performed mine by something like 15% off the shelf, with no fine-tuning or clever ideas. I wished I'd used it from the start.

The difference is likely in that it uses a fiber-based model rather than promises at the execution layer, but I'm not sure.

I like Node.JS streams. It's very satisfying to rent a 250MB memory machine and let it process GB's of data using streams.

The real win here isn’t just performance,it’s convergence.

When ReadableStream behaves the same in the browser, Workers, and other runtimes, stream-based code becomes portable and predictable. That reduces subtle backpressure bugs and eliminates “works here but not there” edge cases.

Standardization at the streams layer is a big deal for building reliable streaming systems across environments.

Yep, it isn't just x; it's y

https://socketcluster.io/ has had such stream implementation and backpressure management since at least 2019.

Here's the WritableConsumableStream module:

https://github.com/SocketCluster/writable-consumable-stream

SocketCluster solves the problem of maintaining message order with async processing.

This feature is even more useful now with LLMs as you can process data live, transform streams with AI with no risk of mangling the message order.

I may have been the first person to use a for-await-of loop in this way with backpressure. At least on an open source project.

the pull-stream module and its ecosystem is relevant here

the idea is basically just use functions. no classes and very little statefulness

https://www.npmjs.com/package/pull-stream

There's a lot I like about this API, mainly the pull-based iterator approach. I don't really see what the value of the sync APIs are though. What's the difference of just using iterators directly for sync streams?

It avoids the overhead of Promises, so I can imagine that this would be quite useful if you know that blocking the thread is fine for a little while (e.g. in a worker).

I mean the APIs like `Stream.pullSync` you could do that with a regular (non-async) iterator/generator.

As a maintainer on the Ky team, I give a big thumbs up to this proposal.

We have run into many problems with web streams over the years and solving them has always proven to be hairy, including the unbounded memory growth from response.clone().

The Deno team implemented a stream API inspired by Go, which I was happy with, until they ultimately acquiesced to web streams.

This proposal shares some of those principles as well.

I really enjoyed reading this article however I can't help but feeling that if you need anything described within it probably shouldn't be writing JS in the first place

There are many use cases where having a value stream is very useful. I do agree having a separate simpler byte only stream would make sense though. I think the current capabilities of web streams should be kept and an IOStream could be added for optimizing byte streams.

Ideally splitting out the use cases would allow both implementations to be simpler, but that ship has probably sailed.

What's wrong with a `Read` `Write` interface like every other language?

    const buffer = new UInt8Array(256)
    const bytesRead = await reader.read(buffer)
    if (bytesRead === 0) {
      // Done
      return
    }

Promises should not be a big overhead. If they are, that seems like a bug in JS engines.

At a native level (C++/rust), a Promise is just a closure added to a list of callbacks for the event loop. Yes, if you did 1 per streamed byte then it would be huge but if you're doing 1 promise per megabyte, (1000 per gig), it really shouldn't add up 1% of perf.

In Rust, a Future can have only exactly one listener awaiting it, which means it doesn't need dynamic allocation and looping for an arbitrary number of .then() callbacks. This allows merging a chain of `.await`ed futures into a single state machine. You could get away with awaiting even on every byte.

I'm fairly sure it's not Promises that are actually the heavy part but the `await` keyword as used in the `for await` loop. That's because await tries to preserve the call stack for debugging, making it a relatively high-level expensive construct from a perf perspective where a promise is a relatively low-level cheap one.

So if you're going to flatten everything into one stream then you can't have a for loop implementation that defensively awaits on every step, or else it'll be slooooooooow. That's my proposal for the change to the language is a syntax like

  for await? (value of stream) {
  }

Async call stacks is an optional feature when the devtools is open. There shouldn't be overhead from await like that?

It's awfully hard to know, and I am not myself sure.

I've only used them in dotnet, I would be very interested to read any strong opinions about the use of Streams both in practice and as an abstract point in API design.

Streams are how modern operating systems work, most commonly to transfer audio, video, file system, and network data from hardware to channels available for applications. So a common scenario is to stream data from a file and pipe it to a network interface for transfer to other computers or to a web browser.

> The problems aren't bugs; they're consequences of design decisions that may have made sense a decade ago, but don't align with how JavaScript developers write code today.

> I'm not here to disparage the work that came before — I'm here to start a conversation about what can potentially come next.

Terrible LLM-slop style. Is Mr Snell letting an LLM write the article for him or has he just appropriated the style?

Heh, I was using emdashes and tricolons long before LLMs appropriated the style but I did let the agent handle some of the details on this. Honestly, it really is just easier sometimes... Especially for blogs posts like this when I've also got a book I'm writing, code to maintain etc. Use tools available to make life easier.

I think you'd be much better served by writing something rough that maintains your own voice!

I'm not sure any emdash use at all is what people are calling out typically(maybe it is?), more the sheer number of them typical in LLM written stuff.

Just ctrl-f'ing through previous public posts, I think there were a total of 7 used across about that many posts. This one for example had 57. I'm not good enough in proper English to know what the normal number is supposed to be, just pointing that out.

Just want to raise my hand and say I too have been using em dashes for considerably longer than LLM has been on every hacker's lips. It's obviously not great being accused of being an AI just because one has a particular style of writing...

I found your article both interesting and readable.

It doesn't really matter what tools are used if the result is good

People are understandably a bit sensitized and sceptical after the last AI generated blog post (and code slop!) by Cloudflare blew up. Personally I'm fine with using AI to help write stuff as long as everything is proof-read and actually represents the authors thoughts. I would have opted to be a bit more careful and not use AI for a few blog posts after the last incident though if I was working at Cloudflare...

What was it specifically about the style that stood out as incongruous, or that hindered comprehension? What was it that made you stumble and start paying close attention to the style rather than to the message? I am looking at the two examples, and I can't see anything wrong with them, especially in the context of the article. They both employ the same rhetorical technique of antithesis, a juxtaposition of contrasting ideas. Surely people wrote like this before? Surely no-one complained?

The problem is less with the style itself and more that it's strongly associated with low-effort content which is going to waste the readers time. It would be nice to be able to give everything the benefit of the doubt, but humans have finite time and LLMs have infinite capacity for producing trite or inaccurate drivel, so readers end up reflexively using LLM tells as a litmus test for (lack of) quality in order to cut through the noise.

You might say well, it's on the Cloudflare blog so it must have some merit, but after the Matrix incident...

I find it more amusing that the benchmarks claim 530 GB/s throughput on an M1 Pro which has a 200GB/s memory bandwidth. The 275 GB/s figure for chained transforms has the same problem.

I suspect the benchmarks, if not most of this project, was completely vibecoded. There are a number of code smells, including links to deleted files, such as https://github.com/jasnell/new-streams/blob/ddc8f8d8dda31b4b... an inexistent REFACTOR-TODO.md

The presence of COMPLETENESS-ANALYSIS.md (https://github.com/jasnell/new-streams/blob/main/COMPLETENES...) isn't reassuring either, as it suggests the "author" of this proposal doesn't sufficiently understand the completeness of his own "work."

These AI signals will die out soon. The models are overusing actual human writing patterns, the humans are noticing and changing how they write, the models are updated, new patterns emerge, etc, etc. The best signal for the quality of writing will always be the source, even if they are "just" prompting the model. I think we can let one incident slide, but they are on notice.

> You might say well, it's on the Cloudflare blog so it must have some merit

I would instead say that it is written by James Snell, who is one of the central figures in the Node community; and therefore it must have some merit.

The idea is well articulated and comes across clear. What’s the issue? Taking a magnifying glass to the whole article to find sentence structure you think is “LLM-slop” is an odd way to dismiss the article entirely.

I’ve read my fair share of LLM slop. This doesn’t qualify.

cloudflare does seem to love ai written everything

You’ve got it backwards: LLMs were trained on human writing and appropriated our style.

Partially true. They've been trained and then aligned towards a preferred style. They don't use em-dashes because they are over-represented in the training material (majority of people don't use them).

It seems likely that with the written word, as with most things, a minority of people produce the majority of content. Most people publish relatively few words compared to professional writers.

Possibly the LLM vendors could bias the models more toward nonprofessional content, but then the quality and utility of the output would suffer. Skip the scientific articles and books, focus on rando internet comments, and you’ll end up with a lot more crap than you already get.

They converge...

It's a real shame that BYOB (bring your own buffer) reads are so complex and such a pain in the neck because for large reads they make a huge difference in terms of GC traffic (for allocating temporary buffers) and CPU time (for the copies).

In an ideal world you could just ask the host to stream 100MB of stuff into a byte array or slice of the wasm heap. Alas.

I wonder if you can get most of the benefit BYOB with a much simpler API:

    for await (const chunk of stream) {
        // process the chunk
        stream.returnChunk(chunk);
    }

(Lately I’ve been thinking that a really nice stream or receive API would return an object with a linear type so that you must consume it and possibly even return it. This would make it impossible to write code where task cancellation causes you to lose received data. Sadly, mainstream languages can’t do this directly.)

BYOB reads definitely add complexity, but the performance gains are significant in memory-sensitive applications. It’s frustrating that managing larger streams isn’t straightforward, especially given the increasing importance of efficiency in JavaScript.

It might be a good idea to look into the research on streams as coalgebras, there is quite a bit, for example here https://cs.ru.nl/~jrot/CTC20/.

Coalgebras might seem too academic but so were monads at some point and now they are everywhere.

I tinkered with an alternative to stream interfaces:

https://github.com/ralusek/streamie

allows you to do things like

    infiniteRecords
    .map(item => doSomeAsyncThing(item), { concurrency: 5 });

    infiniteRecords
    .map(item => doSomeAsyncSingularThing(item), { concurrency: 5 })
    .map(groupOf10 => doSomeBatchThing(groupsOf10), { batchSize: 10 })
    // Can flatten back to single items
    .map(item => backToSingleItem(item), { flatten: true });

Just use AsyncIterator<UIntArray>.

The objection is

> The Web streams spec requires promise creation at numerous points — often in hot paths and often invisible to users. Each read() call doesn't just return a promise; internally, the implementation creates additional promises for queue management, pull() coordination, and backpressure signaling.

But that's 95% manageable by altering buffer sizes.

And as for that last 5%....what are you doing with JS to begin with?

[flagged]

> high-performance data processing tools in JS

I may be naive in asking this, but what leads someone to building high perf data tools in JS? JS doesn't seem to me like it would be the tool of choice for such things

I have a SaaS project where the backend is in JS. I also have some data processing to do with large file (several TB). Doing it is in JS is more convenient as I can reuse code from the backend, and it is also the language I know best.

Performance-wise, I get about half the throughput I had with the same processsing done it rust, which doesn't change anything for my use-case.

However that's not really relevant to the context of the post as I'm using node.js streams which are both saner and fast. I'm guessing that the post is relevant to people using server-side runtimes that only implement web streams.

Browsers are now able to stream files from disk so you can create a high performance tool that'll read locally, do [x] with it and present the results, all without any network overhead.

You don't always have a choice on where you deliver your software. It'd be nice to have good tools wherever you are forced to work.

Browsers

Since when are browsers themselves built in JavaScript? Mainstream, fast ones?

Clarification - in the past when I've written high performance data tools in JS, it was almost entirely to support the use case of needing it to run in a browser. Otherwise, there are indeed more suitable environments available.

To your question, I was about to point out Firefox[1], but realized you clarified 'mainstream'[2]...

[1] https://briangrinstead.com/blog/firefox-webcomponents

[2] https://gs.statcounter.com/browser-market-share

but instead of trying to solve that, this api is just like “too hard no one uses it let’s forget about it”.

right now when i need to wrangle bytes, i switch languages to Golang. it’s easy gc language, and all its IO is built around BYOB api:

interface Reader { read(b: Uint8Array): [number, Error?] }

you pass in your own Uint8Array allocation (in go terms, []byte), the reader fills at most the entire thing, and returns (bytes filled, error). it’s a fully pull stream API with one method at its core. now, the api gets to be that simple because it’s always sync, and blocks until the reader can fill data into the buffer or returns an error indicating no data available right now.

go has a TeeReader with no buffering - it too just blocks until it can write to the forked stream.

https://pkg.go.dev/io#TeeReader

we can’t do the same api in JS, because go gets to insert `await` wherever it wants with its coroutine/goroutine runtime. but we can dream of such simplicity combined with zero allocation performance.

[dead]

[dead]

[flagged]

“ The Streams Standard was developed between 2014 and 2016 with an ambitious goal to provide "APIs for creating, composing, and consuming streams of data that map efficiently to low-level I/O primitives." Before Web streams, the web platform had no standard way to work with streaming data.”

This is what UDP is for. Everything actually has to be async all the way down and since it’s not, we’ll just completely reimplement the OS and network on top of itself and hey maybe when we’re done with that we can do it a third time to have the cloud of clouds.

The entire stack we’re using right down to the hardware is not fit for purpose and we’re burning our talent and money building these ever more brittle towering abstractions.

UDP is a protocol, not an API

True. But it’s also true that trying to shoehorn every use case into TCP streams is counter productive.

A stream API can layer over UDP as well (reading in order of arrival with packet level framing), but such a stream would a bit weird and incompatible with many stream consumers (e.g. [de]compression). A UDP API is simpler and more naturally event (packet) oriented. The concepts don’t mix well.

Still, it would be nice if they browser supported a UDP API instead of the weird and heavy DTLS and QUIC immitations.

TCP or UDP are orthogonal to this, so the original comment feels like a non sequitur. These streams are not network streams and could be a file, chunks of procedural audio, or whatever.

I agree, the stream concept should be (and is) very general and ideally cover all these cases - any “bytes producing” source.

I was trying to be open minded about that and conceive a stream API over a UDP socket. It’d work IMHO, but be a little odd compared to an event-like API.

The browser does have a UDP data stream available for applications to send arbitrary bytes over UDP; it's part of WebRTC.

While Web RTC is built on UDP, it does not allow sending arbitrary UDP. It's DTLS, perhaps encapsulating SCTP, and DCEP.

We're too busy building products while waiting for the perfect system to arrive.

I’m building everything from first principles, I’m not climbing the exponential curve with some billionaire that has to finance it.

I really doubt you are. you're not visiting the transistor shop every time you want to build a react component

Good thing your confidence is a soft requirement :)

For gods sake, finally, somebody have said this!

Well, it's also possible to replace JavaScript with a better language, it's just too late for it...

We deserve a better language than JavaScript.

Sadly it will never happen. WebAssembly failed to keep some of its promises here.

There's always a comment like this in most discussions about javascript.

> WebAssembly failed to keep some of its promises here

classic case of not using an await before your promise

Where can I find these not kept promises?

They haven't yet made languages other than JavaScript first-class languages for the web - https://hacks.mozilla.org/2026/02/making-webassembly-a-first.... I wouldn't call this a broken promise, but it was something people were hoping would take less than a decade.

As wonky as JS is I really like it. Typescript has done such a good job at making it fun to use.

[dead]

The Observables spec should just get merged and implemented.

https://github.com/tc39/proposal-observable

Observables has moved to WHATWG [1] and been implemented in Chrome, although I don't know if the other browsers have expressed any interest (and there's still some issues [2] to be worked through).

But Observables really do not solve the problems being talked about in this post.

[1] https://github.com/WICG/observable [2] https://github.com/WICG/observable/issues/216