KubeDB

By AppsCode

CONTACT SALES

New to KubeDB? Please start here.

Milvus Compute Autoscaling

This guide will show you how to use the KubeDB Autoscaler operator to autoscale the compute resources (CPU/memory) of a Milvus database.

Before You Begin

  • You should be familiar with the following KubeDB concepts:

  • Install the KubeDB Autoscaler operator and a metrics server in your cluster — the autoscaler’s built-in recommender needs metrics to produce recommendations.

    $ kubectl get deploy metrics-server -n kube-system
NAME             READY   UP-TO-DATE   AVAILABLE   AGE
metrics-server   1/1     1            1           5m

  • Complete the dependency setup from Prepare Dependencies. It installs MinIO and creates the object-storage secrets (milvus-storage-config for milvus-standalone, milvus-cluster-storage-config for milvus-cluster).

Note: The yaml files used in this tutorial are stored in docs/guides/milvus/autoscaler/compute/yamls folder in GitHub repository kubedb/docs.

Compute Autoscaling — Standalone Milvus

Deploy the standalone quickstart manifest and wait until it is Ready:

If you deleted a Milvus of the same name in an earlier guide, delete its object-storage prefix first, as shown in Prepare Dependencies. Otherwise the new database can hang in Provisioning.

$ kubectl apply -f https://github.com/kubedb/docs/raw/v2026.10.12/docs/guides/milvus/quickstart/yamls/standalone.yaml
milvus.kubedb.com/milvus-standalone created

$ kubectl get milvuses.kubedb.com -n demo milvus-standalone -w

Then create a MilvusAutoscaler targeting the standalone node:

compute-standalone.yaml

apiVersion: autoscaling.kubedb.com/v1alpha1
kind: MilvusAutoscaler
metadata:
  name: milvus-standalone-compute-autoscaler
  namespace: demo
spec:
  databaseRef:
    name: milvus-standalone
  compute:
    node:
      trigger: "On"
      podLifeTimeThreshold: 10m
      minAllowed:
        cpu: 100m
        memory: 1Gi
      maxAllowed:
        cpu: 1000m
        memory: 2Gi
      resourceDiffPercentage: 20
      controlledResources: ["cpu", "memory"]
  opsRequestOptions:
    apply: IfReady
    timeout: 5m

Here,

  • minAllowed.memory: 1Gi keeps the autoscaler from shrinking Milvus below what it needs. The generated OpsRequest sets the memory limit equal to the request, so a lower floor (for example 256Mi) leaves no headroom and Milvus is OOM-killed under real load.
  • podLifeTimeThreshold: 10m and resourceDiffPercentage: 20 damp the autoscaler. Every OpsRequest it creates restarts the pods, and a freshly started Milvus briefly uses much more CPU. With a short podLifeTimeThreshold and a small resourceDiffPercentage, that startup spike immediately triggers another re-scale (and another restart). Waiting 10 minutes after a pod starts and ignoring differences below 20% prevents this.
$ kubectl apply -f https://github.com/kubedb/docs/raw/v2026.10.12/docs/guides/milvus/autoscaler/compute/yamls/compute-standalone.yaml
milvusautoscaler.autoscaling.kubedb.com/milvus-standalone-compute-autoscaler created

The autoscaler runs its own built-in recommender (it does not create a VerticalPodAutoscaler object). Once the recommender produces a recommendation that differs from the current resources by more than resourceDiffPercentage, the autoscaler creates a VerticalScaling MilvusOpsRequest.

Each autoscaler-created OpsRequest restarts the pods of the role it scales (it uses Restart mode). For a standalone Milvus, that means a short outage every time the autoscaler re-scales.

$ kubectl get milvusautoscaler -n demo
NAME                                   AGE
milvus-standalone-compute-autoscaler   59s

The recommender (fed by the metrics server) records its recommendation in the autoscaler status. Once enough samples are collected, the RecommendationProvided condition becomes True and a target resource set is published. The status has this shape (values and the large checkpoints array elided):

$ kubectl get milvusautoscaler milvus-standalone-compute-autoscaler -n demo -o jsonpath='{.status}' | jq .
{
  "checkpoints": [ ... ],
  "conditions": [ ... ],
  "vpas": [
    {
      "conditions": [
        { "type": "RecommendationProvided", "status": "True", ... }
      ],
      "recommendation": {
        "containerRecommendations": [
          {
            "containerName": "milvus",
            "lowerBound": { "cpu": ..., "memory": ... },
            "target": { "cpu": ..., "memory": ... },
            "uncappedTarget": { "cpu": ..., "memory": ... },
            ...
          }
        ]
      },
      "vpaName": "milvus-standalone"
    }
  ]
}

To see just the recommended target:

$ kubectl get milvusautoscaler milvus-standalone-compute-autoscaler -n demo -o jsonpath='{.status.vpas[0].recommendation.containerRecommendations[0].target}'

An idle standalone uses far less CPU than its 500m request, so the CPU target is lower, while the memory target is held at minAllowed (1Gi). When the recommendation differs from the current request by more than resourceDiffPercentage (20%) and stays within minAllowed/maxAllowed, the autoscaler creates a VerticalScaling MilvusOpsRequest. This is recorded in the autoscaler status as a CreateOpsRequest condition:

$ kubectl get milvusautoscaler milvus-standalone-compute-autoscaler -n demo \
    -o jsonpath='{.status.conditions[?(@.type=="CreateOpsRequest")].message}'
Successfully created MilvusOpsRequest demo/mvops-milvus-standalone-xqwkhv

$ kubectl get milvusopsrequest -n demo
NAME                             TYPE              STATUS        AGE
mvops-milvus-standalone-xqwkhv   VerticalScaling   Progressing   2s

The Ops-manager then applies the vertical scaling exactly as in the vertical scaling guide, right-sizing the pod to the recommended resources.

Compute Autoscaling — Distributed Milvus

For a distributed Milvus, the autoscaler is keyed by role (proxy, mixcoord, datanode, querynode, streamingnode). The sample below enables compute autoscaling for all five roles.

Deploy the distributed quickstart manifest and wait until it is Ready:

$ kubectl apply -f https://github.com/kubedb/docs/raw/v2026.10.12/docs/guides/milvus/quickstart/yamls/distributed.yaml
milvus.kubedb.com/milvus-cluster created

$ kubectl get milvuses.kubedb.com -n demo milvus-cluster -w

compute-distributed.yaml

apiVersion: autoscaling.kubedb.com/v1alpha1
kind: MilvusAutoscaler
metadata:
  name: milvus-compute-autoscaler
  namespace: demo
spec:
  databaseRef:
    name: milvus-cluster
  compute:
    proxy:
      trigger: "On"
      podLifeTimeThreshold: 10m
      minAllowed:
        cpu: 100m
        memory: 1Gi
      maxAllowed:
        cpu: 1000m
        memory: 2Gi
      resourceDiffPercentage: 20
      controlledResources: ["cpu", "memory"]
    mixcoord: { ... }
    datanode: { ... }
    querynode: { ... }
    streamingnode: { ... }
  opsRequestOptions:
    apply: IfReady
    timeout: 5m

(Each role block is identical to the proxy block above; see the full file in the yamls folder.)

$ kubectl apply -f https://github.com/kubedb/docs/raw/v2026.10.12/docs/guides/milvus/autoscaler/compute/yamls/compute-distributed.yaml
milvusautoscaler.autoscaling.kubedb.com/milvus-compute-autoscaler created

The behavior is identical to standalone, except a resource recommendation is produced per role (one status.vpas[] entry per role, for example vpaName: milvus-cluster-proxy). When any role’s recommendation differs from its current resources by more than resourceDiffPercentage, the autoscaler creates a VerticalScaling MilvusOpsRequest scoped to that role (see the vertical scaling guide for what that ops request looks like). The OpsRequests run one at a time, and each restarts the pods of its role.

Cleaning up

The OpsRequests created by a MilvusAutoscaler are owned by it, so deleting the autoscaler also deletes them, including one that is still running. An OpsRequest deleted mid-flight can leave the database paused or with pods that don’t match its spec. Before deleting the autoscaler, wait until none of its OpsRequests is Progressing:

$ kubectl get milvusopsrequest -n demo

Then clean up:

$ kubectl delete milvusautoscaler -n demo --all
$ kubectl patch -n demo milvus.kubedb.com milvus-standalone -p '{"spec":{"deletionPolicy":"WipeOut"}}' --type="merge"
$ kubectl patch -n demo milvus.kubedb.com milvus-cluster -p '{"spec":{"deletionPolicy":"WipeOut"}}' --type="merge"
$ kubectl delete milvus.kubedb.com -n demo milvus-standalone milvus-cluster
$ kubectl delete ns demo

Next Steps

What's on this Page

Search
Improve This Page