An Analysis of CPU Utilization and Service Latency Based on Cool Down Time Configuration in Kubernetes
Annual Conference of KIPS (ACK 2024)
- Silver Award
Summary
Retaining replicas longer can improve readiness but uses more resources. The study varies Cool Down Time and per-pod CPU allocation to examine where latency gains diminish.
- Role
- Second authorundergraduate research mentor
- Outcome
- Longer CDT reduced latency in the slowest 5% of requests, with diminishing gains and greater allocation. ACK 2024 undergraduate paper Silver Award.
Latency and Resource Use as Replicas Scale In
Releasing replicas after a brief drop in demand means preparing them again when requests return. Keeping them longer improves readiness but retains unused resources. This study examined how Cool Down Time (CDT) changes response latency and resource use.
The same replica count can handle different loads depending on per-pod CPU allocation. We varied both CDT and CPU allocation and compared their effects on latency and resource use.
Varying Cool Down Time and CPU Allocation
The cluster ran Ubuntu Server 22.04 with one master and two workers, each with a 2-core 3.1GHz AMD EPYC CPU and 4GB of memory. The PHP web application used HPA with a 70% CPU Threshold and a range of 1–10 replicas.
The load ranged from 25 to 300 GET requests per 15 seconds. Per-pod CPU allocation was set to 0.1, 0.2, 0.4, or 0.5 cores. At each allocation, CDT varied from 60 to 1000 seconds while latency and resource use were measured.
Within each fixed per-pod CPU allocation, CDT was varied to examine the effect of retaining replicas longer. At the same CDT, different CPU allocations exposed differences in per-pod processing capacity. The two variables were examined through separate combinations of controlled settings.
Measuring slow-request latency alongside mean CPU allocation
Response performance was compared through the distribution of the slowest 5% of requests. Mean latency alone can conceal requests waiting while replicas are prepared again.
The resource metric is mean CPU allocation to running replicas. Even at the same per-pod allocation, retaining replicas longer increases aggregate mean allocation. The evaluation compares this additional allocation with the latency improvement.
Diminishing Latency Gains Despite Increasing Resource Use
Longer CDT reduced latency for the slowest 5% of requests and improved SLO attainment, but the gains diminished beyond a certain point. Mean CPU allocation continued to rise as replicas were retained longer. Allocation with 0.5 cores per pod was up to 4.89 times that with 0.1 cores.
When latency changes little despite retaining CPU resources longer, the marginal performance gain has diminished. A very short CDT, however, requires replicas to be prepared again when requests return. The results call for considering response requirements together with allocation between these two conditions.
The conclusion calls for evaluating latency and allocation together for the service load and per-pod resource size, rather than choosing a single fixed CDT in isolation. I contributed as a coauthor while mentoring an undergraduate researcher. The paper received the Silver Award in the ACK 2024 undergraduate paper competition.