An Analysis of Service Latency Based on Replica Resource Granularity in Kubernetes Environment
Annual Conference of KIPS (ACK 2024)
The same total CPU budget can perform differently depending on per-pod allocation. We divided one CPU core among 1–16 replicas to isolate the effect of resource granularity.
- Role
- Co-author · undergraduate research mentor
- Outcome
- Splitting one replica into two raised P95 by 220.58% and P99 by 660.56% in the CPU-bound experiment.
Background
HPA spreads load by adjusting the number of replicas (pods), and that count is inversely proportional to the base resource assigned to each pod. So even at the same total resource, performance depends on how finely that resource is split — the resource granularity. Finer granularity (many small pods) wastes less at low load, but raises the base overhead of running the app in each pod and increases the interference between processes sharing the same physical resource, along with the load on the host OS kernel.
Beyond scaling timing, the study examines how many pods should share the same CPU budget. It holds total allocation constant while varying per-pod resources and replica count.
Experiment
On a one-master, one-worker cluster (2-core 3.1GHz AMD EPYC, 4GB), one physical CPU core was split fairly across 1 / 2 / 4 / 8 / 10 / 16 replicas (from 1 core down to 0.063 cores per pod) to vary the granularity.
The application was an Apache web server running a CPU-intensive trigonometric computation; a load generator sent 300 HTTP requests per minute over 300 s while latency and CPU utilization were measured.
Results
With total CPU held constant, adding replicas could still increase latency. Splitting one replica into two raised P95 and P99 by 220.58% and 660.56%. Mean latency generally increased, but some metrics were similar or lower between eight and ten replicas.
Container overhead and inter-process contention were proposed explanations for the higher latency. Their individual contributions were not isolated and would need separate measurement.
CPU utilization also increased with finer allocation. In this CPU-bound workload, more small pods required more CPU to serve the same load. Choosing per-pod resources therefore required checking latency as well as total allocation. I participated as a co-author and undergraduate research mentor.