cheesepath
pomagrenate/cheesepath
Overview
An ultra-fast, lightweight, all-in-one AI agent framework written in pure Go. Combines linear pipelines, cyclic state graphs, ReAct agent loops, parallel super-steps, time-travel checkpointing, and callbacks into a single compiled runtime with minimal memory footprint.
Technologies & Concepts
Project Analysis
🔴 Problem
Multi-package fragmentation: When building local agent workflows, I found that popular Python ecosystems frequently require installing multiple separate, heavyweight libraries—one for linear prompt/chain composition and another separate framework for cyclic state graphs. Managing disparate abstractions, overlapping dependencies, and bloated virtual environments created friction and made small deployments unwieldy.
Severe memory footprint (RAM bloat): Python-based agent runtimes often require dozens to hundreds of megabytes of resident memory just to import dependencies and initialize the runtime. In high-concurrency production deployments (e.g. running 1,000 to 10,000 concurrent agent workflows) or resource-constrained edge containers, memory usage escalates rapidly into gigabytes (2–4 GB+), creating severe Out-Of-Memory (OOM) risks.
Framework-level scheduling overhead: Interpreted dynamic typing, deep call stacks, and runtime JSON serialization impose non-trivial overhead on every node transition (often 600 µs to 1 ms per node). In multi-agent DAGs or multi-turn ReAct loops with 20+ tool iterations, pure framework tax can add 30 to 100 ms of latency per request—even before accounting for LLM inference time.
Weak compile-time safety: Untyped state dictionaries mean schema mismatches, type errors, or typos in state channel keys only surface at runtime, often deep into a multi-turn agent conversation.
Cold start latency: In containerized serverless or CLI environments, importing heavy multi-package agent libraries requires 200–500 ms just for initialization, making snappy CLI or on-demand execution difficult.
⚪ Baseline
Starting point: Mainstream Python-based agent framework stack (LangGraph 1.2+ alongside LangChain running on Python 3.11 with asyncio), evaluated on the exact same physical machine (Intel Core i5-4310U @ 2.00GHz, Windows 11 AMD64) using Zero-Network Mock Isolation (stubbed 0ms LLM calls to measure pure framework tax).
Baseline characteristics:
- Node transition latency: 666.49 µs per step transition
- 50-Node deep pipeline: 33.32 ms mean traversal, with a p99 tail latency of 118.60 ms, sustaining ~30.0 traversals/sec
- 20-Turn ReAct loop: 27.23 ms mean completion, with a p99 tail of 113.20 ms, sustaining ~36.7 runs/sec
- Concurrency ceiling: Single-process asyncio event loop saturates at ~205 requests/sec, with 5,000+ concurrent workflows consuming an estimated 2.5–5 GB+ RAM
- Deployment size: >850 MB virtual environment on disk, ~250 ms cold start import latency
🔵 Change
Built Cheesepath in pure Go: Created a lightweight, all-in-one AI agent framework using Go 1.23 standard library only (zero CGO, zero external dependencies).
Key architectural changes:
- All-In-One Unified Primitives: Consolidated linear chains (
chain.LLMChain,chain.Sequential), cyclical state graphs (graph.StateGraph[S any]), prebuilt ReAct agents, and memory buffers into a single cohesive library. No need to install multiple disjoint frameworks. - Type-Safe StateGraph via Go Generics: Used Go generics to enforce compile-time type safety across all graph channels, nodes, and reducers. State schema mismatches are caught at build time rather than failing at runtime.
- Parallel Goroutine Super-Steps: Replaced heavy thread pools and dynamic event loops with lightweight Go goroutines (~2 KB initial stack). Branching nodes execute concurrently with microsecond-level scheduling overhead (~5 µs per transition).
- State Channels & Reducers: Implemented channel-based state propagation with composable reducers (
Overwrite,Append, or custom user functions) to safely merge concurrent branch updates. - Time-Travel Checkpointing & HITL: Built thread-scoped snapshot interfaces (
MemorySaver,FileSaver) supporting time-travel state replay and human-in-the-loop (HITL) node interrupts for approval workflows. - Built-In Reliability & Callbacks: Integrated OpenInference-compliant tracing spans, exponential backoff retries with jitter, and fallbacks directly into runnable abstractions.
🟣 Measurement
Test environment: Intel Core i5-4310U @ 2.00GHz (2 cores / 4 threads), 8 GB RAM, Windows 11 AMD64, under Zero-Network Mock Isolation (mock LLM & tool stubs returning 0ms synthetic outputs) to measure pure framework execution overhead.
Benchmark suite:
- Scenario A (Deep Pipeline): 50 sequential nodes evaluating pure step transition latency and DAG execution overhead.
- Scenario B (Cyclic ReAct Loop): 20 consecutive tool-calling cycles with JSON schema parsing and channel reducer merges.
- Scenario C (Concurrency Saturation): Multi-agent triage workflows scaling from 100 to 1,000, 5,000, and 10,000 simultaneous concurrent executions.
- Scenario D (Nested Subgraphs): 3 nested state graphs with conditional routing and dynamic state propagation.
Metrics collected: Step transition latency (µs), mean execution duration (ms), p50/p99 tail latencies, throughput (workflows/sec), peak resident set size (RSS in MB), and cold start / binary size.
🟢 Result
Microsecond node transitions: Step transition latency dropped from 666.49 µs in Python down to 2.33–5.48 µs in Cheesepath (~120x–286x faster).
122x faster 50-node pipeline traversal: Completed the 50-node sequential pipeline in 0.27 ms (vs 33.32 ms), with a p99 tail latency of 1.52 ms (vs 118.60 ms), achieving 3,648 to 8,571 traversals/sec (vs 30.0 ops/s).
48x faster ReAct tool loops: A 20-turn cyclical ReAct loop completed in 0.56 ms (vs 27.23 ms), sustaining 1,259 to 1,775 runs/sec (vs 36.7 runs/sec).
Extreme concurrency scalability (10k flows in 72 MB): At 10,000 simultaneous concurrent workflows, Cheesepath sustained 55,580 req/s (completing all 10k workflows in 179.9 ms total wall time) while consuming only 72.2 MB peak RAM. In contrast, the Python baseline saturated at ~205 req/s and was unviable at 5,000+ flows due to severe OOM risks (>2–4 GB).
47x smaller deployment footprint: Self-contained compiled binary of 18 MB with an instant cold start of <1 ms (vs 850 MB virtual environment and 250 ms import latency in Python).
Performance Visualizations & Empirical Charts
1. Architectural Advantage Overview Dashboard
Four-panel summary comparing Cheesepath (Go) and Python-based frameworks: execution latency reduction across nodes and pipelines, workflow throughput scaling (up to 285x higher), memory consumption under high concurrency (72 MB vs 6 GB+), and deployment artifact footprint (18 MB vs 850 MB with sub-millisecond cold start).
2. Execution Latency Comparison (Logarithmic Scale)
Pure framework overhead across four workload scenarios under zero-network mock isolation. Single node step transitions take only 2.33 µs (vs 666.5 µs), 50-node deep pipelines complete in 0.12 ms (vs 33.3 ms), and 20-turn ReAct loops finish in 0.79 ms (vs 27.2 ms).
3. Throughput Scalability (Workflows / Second)
Throughput comparison across 50-node pipelines (8,571 ops/s vs 30 ops/s), 20-turn ReAct loops (1,259 runs/s vs 36.7 runs/s), nested subgraphs (20,405 runs/s vs 169 runs/s), and concurrent triage (47,699 req/s vs 205 req/s).
4. High-Concurrency Stress Test & Resident Memory Scaling
Saturation curve and RAM footprint scaling from 100 to 10,000 simultaneous workflows. Lightweight Go goroutines maintain a flat, predictable memory profile (44.2 MB at 1,000 flows, 72.2 MB at 10,000 flows), whereas Python asyncio workflows encounter severe OOM risks above 5,000 flows.
5. Enterprise Production Reliability Scorecard
Radar comparison evaluating key production multi-agent criteria: transactional idempotency, self-healing workflows with jitter retries and fallbacks, active reflection guardrails, OpenInference telemetry, concurrency memory efficiency, and microsecond transition latency.
🟡 Lesson
All-in-one cohesion beats fragmented multi-library setups: I initially thought separating chains from graphs was a clean modular approach. In practice, needing to install and coordinate two large, separate frameworks added mental overhead and dependency bloat. Unifying chains, cyclical state graphs, and ReAct loops into one library made building agents much simpler and more enjoyable.
Go generics are a natural fit for stateful graphs: In dynamically typed Python runtimes, state channel schemas are validated at runtime or not at all, which led to silent bugs midway through multi-turn agent conversations. Using Go generics (StateGraph[S any]) caught state mismatches at compile time while preserving full flexibility for custom reducers.
Goroutines completely changed concurrency scaling: I was astonished to see 10,000 simultaneous workflows finish in 180 ms while consuming only 72 MB of RAM. Because Go goroutines start with just 2 KB of stack space, I could run thousands of parallel agent super-steps without worrying about process memory limits or event loop starvation.
Framework overhead matters even when LLMs are slow: People often assume that because LLM network requests take 500 ms, framework overhead doesn't matter. But in high-throughput multi-agent triage, local edge LLM setups, or multi-turn agent loops with 20+ tool iterations, saving 25–30 ms per turn and hundreds of megabytes of RAM makes a massive difference in operational reliability and hosting cost.
Reliability requires first-class retry and callback primitives: An agent framework is only as good as its failure handling. Adding exponential backoff retries with jitter and OpenInference-compatible callback tracing directly into the runnable core made real-world agent behavior predictable instead of flaky.
What I want to explore next: I want to explore compiling Cheesepath to WebAssembly (WASM) so stateful agent graphs can run directly inside browser runtimes, investigate integrating local embedded C++ engines (like PomaiDB) directly via CGO-free IPC, and implement distributed checkpoint backends for long-lived agent sessions.