Skip to content
← Publications
In Preparation 2026

ARTT-HPA: Heterogeneity-Aware Autoscaling with Adaptive Resilient Thresholding and Traffic Allocation for Multi-Replica Microservices on Kubernetes (tentative title)

Donggyun Kim, Hokun Park, Heonchang Yu

Summary

Even with equal traffic shares, replicas on nodes with relatively lower performance can reach overload sooner. ARTT-HPA adjusts the Threshold from workload trend and volatility and allocates traffic according to each replica’s processing capacity.

Role
Co-authorheterogeneous autoscaling research
Outcome
75.64% lower mean P99 than default HPA on NASDAQ; fewest timeouts on both evaluated workloads. In preparation.
ARTT-HPA architecture: autoscaler, Kubernetes, and traffic-scheduler layers
An overview of the ARTT-HPA architecture

Node Overload That Persists After Scale-Out

ARTT-HPA extends LARE-HPA to heterogeneous clusters with different node capabilities.

Existing traffic allocation did not account for differences in node performance. Replicas on nodes with relatively lower performance received the same traffic share as other replicas. Serving the same request volume can use more of their CPU capacity, causing them to reach overload sooner.

HPA bases scale-out decisions on average CPU utilization across all replicas. High utilization on individual replicas can therefore be obscured by the average. Near 300 RPS, some replicas reached 100% CPU but continued receiving requests, sharply increasing P99 latency. This motivated coordinating replica counts with traffic shares based on processing capacity.

LARE-HPA’s Threshold adjustment also needed refinement. Gradually rising demand produced a weak volatility signal, delaying adaptation. On the non-seasonal NASDAQ workload, cumulative SLO attainment fell below default HPA. The follow-up separates workload trend from short-term variation when adjusting the Threshold and coordinates this with traffic allocation.

Separating Workload Trend from Volatility

Trend describes the sustained direction of request volume; volatility describes fluctuations around it. Separate weights let the Threshold respond to gradual growth and short bursts independently.

RANSAC replaces OLS so that a few extreme observations have less influence on the estimated trend. Median absolute deviation (MAD), which measures dispersion around the median, sets the tolerance for identifying outliers.

RANSAC generates candidate regression lines and counts observations within a tolerance derived from MAD. It selects the candidate explaining the most observations, then refits the line on that set. This reduces the influence of isolated extreme values on the overall slope.

The regression slope is the trend indicator, while mean absolute residuals provide the basis for volatility. This separates steady growth with little variation around the fitted line from a similar average trend with large short-term fluctuations.

Thresholds That Reflect CPU Usage per Request

The same increase in requests can require different amounts of CPU across nodes. Nodes with relatively lower performance may use more CPU to serve the same volume. Multiplying volatility by CPU utilization per request incorporates this difference into the Threshold calculation.

Comparing current indicators with their usual levels requires updated means and variances. Welford’s online algorithm incorporates each observation, while only values within bounds based on standard-normal quantiles update the normalization limits. Volatility has a fixed lower bound of zero; trend, which represents both growth and decline, updates both limits.

Each indicator is normalized to 0–1 using bounds updated online. Volatility is nonnegative, while trend represents both growth and decline, so their ranges are maintained separately. Updating those ranges as workload history grows prevents early observations from fixing all later decision boundaries.

Recent changes in the normalized trend and volatility determine their dynamic weights. The more prominent change receives greater weight; a larger combined signal lowers the Threshold for earlier scale-out, while stable demand raises it. The experiment used a 30-observation window and Thresholds from 50% to 100%.

Traffic Shares Based on Next-Interval Load Forecasts

Current active-request counts do not fully describe how much traffic each replica can handle in the next interval. Holt double-exponential smoothing predicts next-interval CPU utilization, allowing the scheduler to assign less traffic to replicas with higher forecast load.

Holt’s model updates the current CPU load level separately from its trend. Adding those components gives the next-period forecast, allowing replicas with the same current utilization but rising and falling loads to receive different predictions.

Holt’s smoothing coefficients determine how strongly recent observations affect the forecast. Larger values respond quickly but are sensitive to noise; smaller values respond more slowly. A Kalman filter compares predicted-state uncertainty with observation noise to calculate the Kalman gain used for the level and trend coefficients. Observation noise is updated through an EWMA of squared prediction residuals, allowing both coefficients to adapt as recent errors change.

Limiting Share Oscillation and Initializing New Replicas

Applying the full load difference to traffic shares can shift overload to the opposite side in the next interval. A sigmoid bounds each adjustment to reduce repeated oscillation.

The difference between predicted mean CPU load for the component and each replica’s forecast determines a correction factor. The experiment bounded this sigmoid factor to 0.5–2. Existing traffic proportions are multiplied by the factor and normalized to sum to one, so the correction factor is distinct from the final traffic share.

New replicas created during scale-out lack observation history. A warm-start uses initial values from a replica in the same component with sufficient history. This avoids assigning too little traffic simply because measurements are missing and provides values for the first forecast. The module communicates through the Kubernetes API without changing application code.

Evaluation with seasonal and non-seasonal workloads

The evaluation used one master and five workers with different processing capabilities, running Kubernetes 1.27.5, Istio 1.23.2, and Prometheus 3.2.1. Five microservices sequentially called matrix multiplications of different sizes. ARTT-HPA was compared with default HPA, LARE-HPA, and Istio Least Request on NASA-HTTP and NASDAQ workloads under a 5-second SLO.

NASA-HTTP represents a seasonal workload, while the request pattern derived from NASDAQ data represents a non-seasonal one. Threshold adjustment and traffic allocation are evaluated together under different autocorrelation characteristics rather than only one repeating pattern.

Latency is reported as the mean of interval-level P99 values, while failures are counted separately as five-second timeouts. CPU utilization is measured alongside them to compare response quality and how fully allocated resources are used.

75.64% Lower Mean P99 on NASDAQ

On NASDAQ, mean P99 fell from 4,180.0ms with default HPA, 3,864.0ms with Least Request, and 4,476.2ms with LARE-HPA to 1,018.3ms with ARTT-HPA: reductions of 75.64%, 73.65%, and 77.25%. ARTT-HPA had the fewest 5-second timeouts on both workloads.

On NASA-HTTP, Least Request had a lower mean P99: 1,139.8ms versus 1,763.5ms for ARTT-HPA. However, it had 5,364,327 timeouts versus 4,391,250 for ARTT-HPA. Failed requests can be excluded from latency calculations, so the evaluation considered latency for completed requests alongside timeout counts.

Average component CPU utilization across the workloads was 50.22%, compared with 32.50% for default HPA, 46.43% for Least Request, and 36.80% for LARE-HPA. ARTT-HPA made greater use of allocated CPU while reducing latency and timeouts. The study is being prepared for conference submission; its title is tentative.

Work in preparation · Table 2. Mean P99 averages interval P99 values; timeout counts are reported separately.
WorkloadMethodMean P99 (ms)Timeouts
NASA-HTTPHPA3,854.033,196,096
NASA-HTTPLeast Request1,139.85,364,327
NASA-HTTPLARE-HPA3,248.39,436,492
NASA-HTTPARTT-HPA1,763.54,391,250
NASDAQHPA4,180.01,989,924
NASDAQLeast Request3,864.0865,657
NASDAQLARE-HPA4,476.22,109,292
NASDAQARTT-HPA1,018.3386,277