Batching over a list
Run an agent once per item of a list input — fanned out in parallel, with an ordered list of results back.
Batching turns one agent into a fleet. Point the agent at a list input and it runs once per item — concurrently, up to a parallelism you set — and its response becomes a list of the per-item results, in order. There's no separate "loop" node to wire: batching is a property of the agent itself, driven by the shape of its input.
It's the natural partner of anything that produces a list — for example, the Document Splitter chunks a long document, and a batching agent processes every chunk in parallel.
Turning it on
Batching lives in the agent's configuration — in the agent builder and in an agent node's configuration on the canvas. Toggle Run once per item of a list input.
The toggle is available when the agent's input is bound to a list. If nothing in the agent's input resolves to one, the control stays disabled until it does.
Inline agents only
Batching applies to agents configured in place. A node that references a standalone built agent can't batch — the reference runs the agent as-is.
Choosing the list
An agent's input can be a list itself, or an object with a list somewhere inside it. When more than one list is in scope, a List to batch over selector appears:
- The whole input — the input itself is the list; the agent runs once per element.
- A field — one field of the input object is the list. The agent runs once per element of that field, and every other field is passed along unchanged to each item — useful for sending the same context or instructions with every chunk.
Parallelism
Parallel items controls how many items run at once (default 5). Each item is a full agent call of its own, so this is the knob that trades speed against how hard you hit your model provider.
What comes out
The agent's output becomes an ordered list — one entry per input item, in the same order as the input, regardless of which items finished first. If the agent has a structured output, downstream steps see an array of that type, so wiring the result into the rest of a workflow works like any other list.
Two edge cases worth knowing:
- An empty list runs nothing and returns an empty list.
- A failed item fails the execution. If one item can't complete (after automatic retries), the whole node fails rather than silently returning a result with gaps — a quietly missing item is exactly the bug batching exists to eliminate.
In the trace
Batching stays fully observable. On the canvas, a batching agent wears a batch chip. In the execution trace, the agent's row shows a banner with how many items ran and at what parallelism, and each item appears beneath it as its own collapsible entry — Item 1 of N, with its own messages, tokens, cost, and duration. Nothing is averaged away.