A contrast to start. In one cross-comparison of resident coding-agent harnesses, a single Claude Code session occupies 386.6 MB of memory (PSS) [secondhand, jcode README benchmark]; Infinity, open-sourced by AWS on October 23, 2026, keeps a waiting agent at roughly 100 KB [official] — three orders of magnitude apart. How does an agent degrade into "just data" while waiting, and what does the figure measure? This piece takes apart Infinity (Apache-2.0, Rust): mechanism, underpinnings, numbers, and criticism.
Scope: compiled from the official blog, the GitHub repository, and secondhand reviews; not a hands-on deployment test. Figures are tagged [official] or [secondhand]; conflicting counts are listed side by side.
Release and repository: not under the aws org, but under Hydro
On October 23, 2026, the AWS Open Source Blog published "Introducing Infinity: a runtime for extreme agent concurrency," by Shadaj Laddad of the AWS Hydro team [official]. One circulated claim needs correcting: the repository is not under the aws org — aws/infinity and aws/hydro both return 404 in testing. The code lives under hydro-project. Repository address: github.com/hydro-project/infinity. Hydro is a collaborative effort between AWS and UC Berkeley; the blog: "we're releasing Infinity under the Apache 2.0 open source license as part of the Hydro project" [official].
Repository facts (GitHub API, 2026-10-26): Apache License 2.0 (LICENSE.txt at the root), Rust, 25 stars, 54 open issues (an org-page snapshot showed 27 / 30 — snapshots differ). Created 2026-04-13, last pushed 2026-10-23 — still receiving commits on release day. The description: "An agent runtime and tool protocol for principled agent concurrency."
Honesty about traction: 25 stars is quiet for the AWS name, and the only HN thread sits at 2 points, 0 comments [secondhand, Algolia]. AWS's own label matches: "an early research release, and we are sharing it to get early feedback."
The suspension mechanism: agents are expensive while waiting, not while thinking
It starts from an observation about waiting: "Agents spend much of their time waiting on tools. ... While the agent waits, it holds a process and its memory, and it cannot respond to events that arrive in the meantime" — a twenty-minute build, hours until the next pull request, an approval landing tomorrow morning [official].
Infinity's core observation: the moment an agent calls a tool, its conversation history already contains everything needed to continue. "If the runtime persists this history, it can discard all in-memory state, and any process with access to the store can resume it later" [official]. In order: dispatch the tool call, persist the history, discard all memory state, suspend; any process that reaches the store resumes the agent when the result arrives.
Technically this resembles a stackless coroutine: an OS thread keeps its entire call stack while blocked; a stackless coroutine (Rust's async fn) compiles into a state machine retaining only values relevant at each await point, driving the cost of waiting toward zero [official]. For an agent, the conversation history is the continuation. The README: "Infinity does for agents what async did for threads: instead of blocking on slow tools, Infinity agents run them concurrently, yield while they wait, and cost nothing until the next event arrives."
In execution, Infinity runs each agent as bounded slices: load the history from a pluggable store, run one model completion, persist the output, dispatch at most one tool call, end the slice [official]. A tool call is the await point; a slice never blocks. Agents are expensive while waiting, not while thinking — Infinity cuts waiting to nearly nothing.
The serverless underpinnings and the injected tool call
Once agents suspend, waiting costs zero compute; the reference deployment lands on serverless. From the blog: "Our reference deployment on AWS uses an SQS FIFO queue as the input queue, with one MessageGroupId per thread. Each delivery invokes an AWS Lambda function that runs one step and exits. Conversation histories are stored in Aurora DSQL and deduplication state in DynamoDB" [official]; SQS delivery delays and EventBridge Scheduler implement timers, with CDK constructs in the repository.
First, the synthetic tool call receive_event__injected: model APIs require every tool call to be matched by exactly one result, so an external event — say a GitHub pull_request/opened payload — has no clean channel to reach the model. Infinity inserts an assistant tool call the model never made, followed by a tool result carrying the event content; arguments include original_tool_name, original_tool_call_id, and original_args to anchor provenance, and the __injected suffix marks runtime injection. The runtime never offers that name as a tool; attempts to call it are rejected [official].
Second, agent-to-agent messaging: a parent thread spawns children with the built-in spawn_thread; a child's report returns as a synthetic event, so it does not flood the parent context. The input queue serializes by thread_id, runs threads in parallel — actor-system semantics [official]. Separately, the README describes the Reactive Agent Protocol (RAP): tool results and subscribed events are delivered as they become ready, even to an agent with no process running; the spec is open (source: README / docs site).
The quality of the numbers: two qualifiers that must survive quoting
Back to 100 KB and 75,000. The full official setup [official]: 80,000 agents, 20 turns each, 3.2 million completions; each turn, the model streams about 1.2 KB and calls a tool, the result returns through the input queue, and a second completion closes the turn. Hardware: single-threaded AMD EPYC 9R14. Storage: in-memory stores. Model: a simulated model — not a real model API.
Two qualifiers, and neither can be dropped when the numbers travel. First, the simulated model: the blog says so explicitly. Real responses are larger, slower, and costlier; long-context behavior was not covered — beneath the "75,000 agents in 8 GB" headline sits a measurement against a simulated model.
Second, the extrapolation. "Resident memory grew linearly, at about 100 KB per idle agent. At this rate, about 75,000 agents fit in 8 GB, which is the memory of a Raspberry Pi 5" [official]. The 75,000 is extrapolated from 100 KB per agent (8 GB ÷ roughly 103 KB ≈ 78,000, rounded by AWS); the test never ran on a Raspberry Pi — the Pi 5 is only an analogy for memory size. The phrasing is "per idle agent," and "Most of this memory is taken up by the conversation histories" — the bulk grows with context length.
One gap AWS did not explain: 1.2 KB × 40 completions ≈ 48 KB, well short of 100 KB; the blog does not break down the difference [unverified]. The project documentation carries a separate figure — roughly 103 KB per agent after 20 tool-calling turns, 8.4 GB for 80,000 resident idle agents — secondhand transcription, not directly verified [secondhand]. 80,000 × 20 × two completions per turn does equal 3.2 million; figures under a real model API and a real database were not published.
Against the resident camp: three orders of magnitude, with caveats
Put Infinity's 100 KB next to resident harnesses' hundreds of megabytes: roughly three orders of magnitude [secondhand, and the two sides measure different things]. The jcode README's nine-harness comparison (PSS): single-session Claude Code 386.6 MB, OpenCode 371.5 MB, Copilot CLI 333.3 MB, Cursor Agent 214.9 MB, Codex CLI 140.0 MB; per-additional-session increments of about 212.7 MB (Claude Code) and 21.6 MB (Codex CLI); roughly 2.3 GB for Claude Code at ten sessions — vendor self-measurement, not independently verified [secondhand]. A probe-based measurement (apiad/aegis issue #415) recorded the resident Claude Code harness at 235–319 MB RSS [secondhand].
Caliber difference first: a resident harness's hundreds of megabytes include the runtime and local state; Infinity's suspended 100 KB is only the conversation history. Not the same object measured two ways — two architectures with different cost structures. A resident agent pays full price for memory while waiting; a suspended agent's waiting is nearly free. The trade: suspended state depends on external durable storage; memory saved returns as storage I/O pressure.
A route comparison in passing: Google Cloud's Agent Compute Engine (ACE), announced October 22, takes the other path — one hardware-isolated microVM per agent, suspended via fast checkpoint/restore when idle [official, each party's own figures]. ACE answers "runs safely"; Infinity answers "waits cheaply." Media commentary suggests production stacks will use both [secondhand].
Shortcomings and criticism: five buckets of cold water first
No large-scale community discussion exists yet (the only HN thread: 2 points, 0 comments), but media reviews have laid out the criticisms:
One, the simulated-model benchmark is not hard evidence. tsnmedia: it measures memory "with a simulated model, on a server," and "the figures are AWS's own and have not been independently checked" [secondhand]. Since conversation histories dominate memory, suspended memory under long contexts will significantly exceed 100 KB.
Two, the Raspberry Pi is an analogy, not a measurement. "the test did not run on one. The Pi is a comparison of memory size" [secondhand]. Any retelling that treats it as measured is wrong.
Three, persistence I/O becomes the new bottleneck. techbytes: dropping idle memory to 100 KB trades it for high-frequency reads and writes against durable storage from tens of thousands of slice start-stops; adopters must verify their database backends sustain the rates [secondhand].
Four, ecosystem mismatch. The mainstream agent ecosystem is Python-centered; Infinity is Rust. Whether Python bindings arrive depends on future releases; for Python-heavy teams the integration cost is real [secondhand].
Five, race conditions do not disappear; they move. stackademic: "Race conditions do not disappear; they move to message ordering guarantees per thread_id"; observability must trace along slice boundaries, not HTTP requests. The engineering bar is nontrivial: pluggable stores, durable queues, idempotent tools, and converting synchronous tool returns into async dispatch with write-back queues [secondhand].
Finally, the absence of market validation is the finding: 25 stars, 54 open issues, zero substantive HN discussion. It has not been tested by real production loads — that is not a flaw, it is the current state.
Getting started: the README quick start
The README quick start, verbatim:
let system = AgentSystemBuilder::new_local(
InMemoryConversationStore::new(),
InMemoryStateStore::new(),
StaticModel::new(provider, "global.anthropic.claude-sonnet-4-6").await?,
).start();
let mut thread = system.thread_builder().launch().await;
thread.send_user_text("Write a haiku about Rust").await?;The README splits the project into three components: the Infinity Runtime (with Lambda serverless deployment docs and CDK constructs), the Reactive Agent Protocol (RAP), and Infinity Code — "A coding agent built on the runtime," which runs builds and tests in the background and delivers diffs. On MCP compatibility: "Existing MCP servers run unchanged through a compatibility layer, and anyone can implement the open specification." Watch the sourcing: the MCP layer and Infinity Code appear only in the GitHub README and documentation site, not in the blog post — cite them separately. Docs site: infinity.hydro.run; repository address again: github.com/hydro-project/infinity.
FAQ
Q1: Is "75,000 agents in 8 GB" a measured result? A1: No — an extrapolation. The benchmark measured roughly 100 KB per idle agent on a single-threaded AMD EPYC 9R14; 75,000 is 8 GB divided by roughly 103 KB, rounded. It used a simulated model with in-memory stores and never ran on a Raspberry Pi — 8 GB is only the Pi 5's memory size [official].
Q2: Where does a suspended agent's state live? Is context lost on resume? A2: In a pluggable conversation store. Suspended, the agent is only its conversation history; any process with store access resumes it. Context is not lost, but that presupposes the store's availability [official].
Q3: Is Infinity an either-or choice with microVM isolation like Google ACE? A3: No. ACE answers "runs safely": one hardware-isolated microVM per agent, snapshot-suspended when idle. Infinity answers "waits cheaply": suspended, the agent degrades into data. Media commentary suggests using both in production — microVM outside, slice scheduling inside [secondhand].
Q4: Can existing MCP servers plug in directly? A4: Per the README, yes: "Existing MCP servers run unchanged through a compatibility layer." The claim comes from the README/docs site, not the official blog; and under the asynchronous slice model, tool authors generally convert synchronous returns into async dispatch with write-back queues [secondhand].
Q5: Is this production-ready? A5: The official positioning is an early research release under Apache-2.0; 25 stars and zero substantive HN discussion mean no market validation yet. Track the repository's push cadence, run PoCs on non-critical paths, and validate latency and I/O under a real model API and a real database — AWS publishes no figures for either.
Sources
[official]
- AWS Open Source Blog: "Introducing Infinity: a runtime for extreme agent concurrency," Shadaj Laddad, 2026-10-23 (motivation, mechanism, slices, benchmark, deployment, positioning)
- GitHub repository hydro-project/infinity (API, 2026-10-26: Apache-2.0, Rust, 25 stars, 54 open issues, created 2026-04-13, pushed 2026-10-23); README; docs site infinity.hydro.run
- Google Cloud Blog: "Introducing Agent Compute Engine and expanding agent governance," 2026-10-22 (ACE comparison)
[secondhand]
- tsnmedia.org (simulated-model critique, Pi analogy clarification); toast.com.cn (Chinese recap); techbytes.app (persistence I/O, Python ecosystem); stackademic.com (race-condition migration, engineering bar, dual-stack view)
- txtmix.com summarizing the jcode README (nine-harness comparison, PSS, vendor self-measurement); github.com/apiad/aegis issue #415 (Claude Code harness 235–319 MB RSS, 2026-10-15)
- datastudios.org (project-documentation figures of 103 KB / 8.4 GB, transcribed, not directly verified); HN Algolia (only related thread: 2 points, 0 comments, 2026-10-24)
Related reading
- Suspending, isolating, or keeping resident: choosing an agent concurrency runtime — the companion route-selection comparison.
- A runtime SOP for sandboxing Claude Code — the companion hands-on guide.
- Five agent isolation schemes compared — an earlier piece (2026-08-19) on Firecracker, CubeSandbox, agent-sandbox, E2B, and Daytona.
Compiled from public sources (repository data as of 2026-10-26) with figures tagged [official] or [secondhand]; a representative survey, not a hands-on deployment test, and not technical selection advice.