An Analysis of CPU Utilization and Service Latency Based on Cool Down Time Configuration in Kubernetes
Annual Conference of KIPS (ACK 2024)
- Silver Award
Keeping replicas longer can help latency at the cost of more resources. We varied both cool-down time and per-pod CPU to identify diminishing latency returns.
- Role
- Co-author · undergraduate research mentor
- Outcome
- Longer CDT reduced latency in the slowest 5% of requests, with diminishing gains and greater allocation. ACK 2024 undergraduate paper Silver Award.
Background
Releasing pods after a brief drop in demand means preparing them again when requests return. Keeping them longer improves readiness but retains unused resources. This study examined how cool-down time (CDT) changes that trade-off.
Comparing waiting times alone was not enough: the same replica count can handle different loads depending on per-pod CPU allocation. We varied both CDT and CPU allocation, measuring request latency alongside resource use.
Experiment
A cluster of one master and two workers (each 2-core 3.1GHz AMD EPYC, 4GB) on Ubuntu Server 22.04 ran a PHP web app, with HPA scaling on 70% CPU between 1 and 10 pods.
Under a load of 25–300 GET requests per 15s, latency and resource usage were measured while sweeping the per-pod CPU allocation (0.1 / 0.2 / 0.4 / 0.5 cores) and the cool-down time (60 s to 1000 s).
Results
A longer CDT lowered the top-5% tail latency and improved SLO satisfaction, but past a point latency no longer dropped meaningfully. Meanwhile a longer CDT kept pods alive longer and raised average CPU usage — with 0.5 cores per pod, resource usage grew up to 4.89× versus 0.1 cores.
A longer CDT alone was not enough: resource use continued to rise after latency gains diminished. Both measurements were needed to choose a setting. I co-authored the paper while mentoring an undergraduate researcher; it received a Silver Award at the ACK 2024 undergraduate paper competition.