InferSuite: A Microarchitectural Characterization and System Profiling of LLM Serving and Agentic Inference - CPU/GPU Time and TMA-Style Breakdown Across Serving and Agentic Workloads

Abstract

Deployed LLM systems are commonly treated as GPU workloads. In practice, retrieval, cache lookup, storage, and agent tool execution also consume CPU and memory resources. It remains unclear where time and machine capacity are spent across the full system. This thesis addresses that question with InferSuite, a deployable LLM service and measurement suite for quantifying time and resource use across retrieval, semantic caching, and model serving. Separately from the service, the same measurement method is applied to two existing agentic workloads, a repository-repair agent and a browser agent, driven through their own unmodified harnesses. Everything runs on one bare-metal workstation and is measured through container-scoped hardware counters, Top-Down Microarchitecture Analysis, and GPU profiling, with every measurement split into what happens during inference and outside it. The two studies give opposite answers. In the service, GPU generation holds 95 to 99 percent of request time, and the largest CPU consumer is the serving engine’s synchronization busy-wait, about two cores that look healthy on every counter while doing no useful work, most of which a blocking wait policy hands back. In the agents, the CPU does the real work outside inference, executing the delegated tool payload and the orchestration harness, whose cost compounds with conversation length. The agent machine averages under two percent utilization of its partition yet briefly demands all of it. On the GPU, quantized matrix kernels dominate, compute-bound in prefill and memory-bound in decode. The serving CPU should therefore wait more cheaply rather than be tuned, agent machines should be provisioned for bursts and shared rather than dedicated, and the harness, the serial term of the agentic loop, is the natural next target for accelerating agentic systems.