Why It’s an Iterator, Not a Channel

A Go LLM adapter, server-sent events, and where streaming conventions actually come from.

I’m wiring OpenAI into Google’s Agent Development Kit, the Go one. ADK is model-agnostic by design, but the Go SDK ships only a Gemini provider (and an Apigee gateway), no OpenAI adapter. So to use another model you implement its model.LLM interface yourself, two methods:

type LLM interface {
    Name() string
    GenerateContent(ctx context.Context, req *LLMRequest, stream bool) iter.Seq2[*LLMResponse, error]
}

Streaming returns iter.Seq2[*LLMResponse, error]. An iterator. I’d been reaching for a channel, a goroutine producing responses and the caller ranging over <-ch. That’s the Go-native way to stream, or so I’d assumed. ADK doesn’t do that. It hands back a range-over-func iterator and expects you to write one.

This is a small API choice with a long shadow. Pick a channel and you’ve smuggled a goroutine, a buffer policy, cancellation behavior, and backpressure into the interface. Pick an iterator and the caller pulls the next response when it is ready. Same tokens on the wire. Different ownership model in the code.

So why an iterator?

The easy answer is wrong

The first explanation is cultural. Streaming LLM APIs grew up in Python, and Python is generators all the way down. DataLoaders, training loops, yield everywhere. The reference SDKs are Python-first, so the feel of it is Python: stream=True, for chunk in stream. The convention leaked downstream from ML culture into every other SDK.

It’s a real description of the surface and a bad explanation of the cause. If Python culture were the reason, the pattern would carry Python’s habits with it wherever it went. It doesn’t, and Go is where you can see it not happen.

The wire is a sequence

Start with the transport. OpenAI’s docs say streaming runs “over server-sent events (SSE).” Anthropic’s say the same. SSE is a lazy sequence of events over HTTP, data: lines arriving one at a time, terminated by a sentinel ([DONE] for OpenAI, message_stop for Anthropic).

Once the wire is a sequence of events, “iterate the events” is the obvious client surface in any language. No Python required to arrive there. The heritage runs back through the web. The SSE decoder in Anthropic’s Python SDK carries a comment that it was “initially copied from httpx-sse.” That’s web-streaming plumbing, not a DataLoader.

But the wire doesn’t force one exact API. The SDKs I checked all preserve the sequence, then expose it through whatever their language already considers normal. openai-node exposes a Stream class with a fromReadableStream helper that consumes newline-delimited JSON, no SSE framing at all. One iterator class, two transports. Anthropic ships .on("text", ...) event emitters next to the iterable. Vercel’s AI SDK gives you onChunk and onFinish callbacks alongside an async iterable on the same result. OpenAI’s Realtime API runs over WebSocket.

The wire makes the sequence natural. The iterator is the convention each language layers on top, because each one already had an iterator sitting there.

Same surface, different plumbing

Here’s the part that convinced me it isn’t a Python thing. Line up the SDKs and the surface is identical everywhere. Get a stream, iterate the chunks. The piece that changes is async coloring.

Python ships both colors from one file:

class Stream(Generic[_T]):       # for chunk in stream
class AsyncStream(Generic[_T]):  # async for chunk in stream

JavaScript picks the async one, for await (const chunk of stream). Go drops the color entirely:

for resp, err := range llm.GenerateContent(ctx, req, true) {
    // handle resp, err
}

No async, no goroutine, no channel. The iter.Seq2 does a blocking pull. Decode the next event, yield the next response, stop when the stream ends. Anthropic’s Go SDK works the same way: for stream.Next() { ev := stream.Current() }, decoding synchronously, no goroutines or callbacks underneath.

This is Bob Nystrom’s What Color Is Your Function playing out in the wild. Python and JS have to color streaming async, because reading a socket without blocking the world means async/await. Go has no such color. A goroutine blocking on a socket read is cheap, so the iterator can block and the concurrency lives around it. Same abstraction across all three. The plumbing underneath follows each language’s concurrency model.

And it isn’t that each language was stuck with one tool. Python has channels too. asyncio.Queue ships in the standard library, Trio and AnyIO add memory channels, the same CSP idea as Go’s chan. A vendor could have handed you a receive channel to range over. The major LLM SDKs didn’t.

Both languages had the channel and the iterator on the shelf. Both reached past the channel for the iterator. If generator culture were imposing the pattern, you’d expect async iterators everywhere it spread. Go kept the sequence and threw away the color.

It was an iterator from day one

I wanted the convention to have a birthday. Some release where a vendor switched from callbacks to iterators and everyone copied it. I didn’t find one.

The OpenAI API launched in June 2020 with no streaming at all. Request, response, done. The first public release of openai-python (v0.2.6, 2020-10-25) is where streaming shows up, and the first line of it was already an iterator. parse_stream is a plain generator over the response lines:

def parse_stream(rbody):
    for line in rbody:
        if line:
            if line == b"data: [DONE]":
                return
            if hasattr(line, "decode"):
                line = line.decode("utf-8")
            if line.startswith("data: "):
                line = line[len("data: ") :]
            yield line

Strip the data: prefix, stop on the [DONE] sentinel, yield the line. Completion.create(stream=True) wraps that in a generator expression and hands it back, so the surface was for chunk in completion from the start. And it was SSE-specific by construction: the HTTP layer only enters streaming mode when the response says so, if stream and "text/event-stream" in result.headers.get("Content-Type", ""): content = result.iter_lines(). No SSE header, no stream. There was no callback era to migrate away from. The first streaming OpenAI ever shipped was a generator over SSE.

The typed rewrite came three years later. In v1.0.0 (2023-11-06) the same iterate-the-chunks mechanism got wrapped in typed Stream[ChatCompletionChunk] / AsyncStream classes with a dedicated SSEDecoder. New types, same surface. The sync/async split predates it too: the pre-1.0 SDK already shipped parse_stream next to an async def parse_stream_async. Same abstraction, different plumbing, showing up inside one SDK before it ever shows up across vendors.

Match the wire, then speak your language

I went looking for the smoking gun. A changelog, a GitHub issue, an engineering blog where someone at OpenAI or Anthropic wrote “we return iterators because that’s how Python streams data.” I didn’t find it. The idiom is real and uniform. The claim that ML culture caused it is a good story I can’t source.

What I can source is the form. The wire is a sequence of events. Every language has had a way to consume a sequence lazily for years (Python generators since PEP 255 in 2001, Go’s range-over-func since 1.23 in 2024). When streaming endpoints arrived, each SDK reached for the primitive it already owned.

The channel I reached for would have worked. It also would have made me choose goroutine ownership, buffering, and cancellation semantics before the adapter had earned that complexity. ADK’s iter.Seq2 is the honest Go version of the same idea.

The OpenAI provider I ended up writing for ADK is a Go iterator wrapping OpenAI’s SSE, the whole convergence in one adapter: OpenAI’s wire, Go’s primitive, no async color in between.

It handed me an iterator where I expected a channel. That wasn’t a Python habit leaking into Go. It was the wire’s shape, and the smaller bet.