An Analysis of Response Time Depending on CPU Resource Contention between Pods in Kubernetes Environment
Annual Conference of KIPS (ACK 2024)
Summary
A service can respond more slowly under the same request load when other work shares its node. Separate experiments vary competing CPU utilization and task count to measure the effect on latency.
- Role
- Second authorundergraduate research mentor
- Outcome
- At 450 requests per 30 seconds, mean latency rose from 51ms to 236ms as competing CPU load increased from 0% to 70%.
Slower Responses Due to Other Pods Under the Same Request Load
Cloud workloads share nodes to improve utilization. However, spare capacity at the node level does not mean every service receives CPU when it needs it.
We measured how competing work on the same node affected the target service, varying competing CPU occupancy and task count separately.
Varying Competing CPU Load and Task Count Separately
An Intel Core i5-12600K system with DDR5 32GiB hosted a VM-based cluster of one master and three workers, each with 2 vCPUs and 4GiB of memory. The progrium/stress image controlled competing CPU load.
The target service was php-apache computing trigonometric functions. HTTP requests arrived from outside the cluster. Separate experiments changed competing CPU occupancy and the number of CPU-intensive tasks while measuring response latency.
Controlled comparisons of CPU scarcity and task count
The first experiment fixed competing tasks at two and varied CPU occupancy from 0% to 70%. At each setting, the target received 150, 300, or 450 requests per 30 seconds, comparing reduced CPU availability with increasing service load.
The second held competing CPU occupancy at 65% and varied task count across 2, 4, 6, 8, and 10. The target received 400 requests per 30 seconds, with mean, 90th, and 95th percentile latency compared to test whether task count changes the tail at the same CPU occupancy.
Mean Response Time Rising from 51ms to 236ms
With 450 requests per 30 seconds and two competing tasks, raising competing CPU load from 0% to 70% increased mean latency from 51ms to 236ms. The paper reports an increase of approximately 361% for this configuration.
A separate experiment held competing occupancy at 65% and varied task count from two to ten. Mean latency changed little, but upper-percentile latency increased, with a reported maximum increase of 19.4%. Context switching and cache/TLB changes were proposed explanations; isolating their individual contributions requires further measurement.
The occupancy experiment examines reduced resources available to the target service; the task-count experiment examines execution among more competing tasks at the same CPU occupancy. Even where mean latency changed little in the second experiment, upper percentiles increased, showing a difference not captured by CPU utilization alone.
The target service received the same request load but responded differently depending on colocated work. This suggests considering workload characteristics alongside available resources when placing pods. I participated as a co-author and undergraduate research mentor.
| Competing CPU load | Mean latency (ms) |
|---|---|
| 0% | 51 |
| 40% | 81 |
| 60% | 153 |
| 70% | 236 |