KubeDB

By AppsCode

CONTACT SALES

New to KubeDB? Please start here.

MilvusAutoscaler

What is MilvusAutoscaler

MilvusAutoscaler is a Kubernetes CustomResourceDefinition (CRD). It provides declarative configuration for autoscaling the compute resources and persistent storage of a Milvus database.

Depending on which section you configure, the autoscaler creates one of these ops requests automatically:

  • VerticalScaling MilvusOpsRequest for compute autoscaling.
  • VolumeExpansion MilvusOpsRequest for storage autoscaling.

Sample MilvusAutoscaler Objects

Standalone

apiVersion: autoscaling.kubedb.com/v1alpha1
kind: MilvusAutoscaler
metadata:
  name: milvus-standalone-autoscaler
  namespace: demo
spec:
  databaseRef:
    name: milvus-standalone
  compute:
    node:
      trigger: "On"
      podLifeTimeThreshold: 10m
      minAllowed:
        cpu: 100m
        memory: 1Gi
      maxAllowed:
        cpu: 1000m
        memory: 2Gi
      resourceDiffPercentage: 20
      controlledResources: ["cpu", "memory"]
  storage:
    node:
      trigger: "On"
      usageThreshold: 30
      expansionMode: "Offline"
      scalingRules:
        - appliesUpto: "100Ti"
          threshold: "50%"
  opsRequestOptions:
    apply: IfReady
    timeout: 10m

Distributed

apiVersion: autoscaling.kubedb.com/v1alpha1
kind: MilvusAutoscaler
metadata:
  name: milvus-cluster-autoscaler
  namespace: demo
spec:
  databaseRef:
    name: milvus-cluster
  compute:
    proxy:
      trigger: "On"
      podLifeTimeThreshold: 10m
      minAllowed:
        cpu: 100m
        memory: 1Gi
      maxAllowed:
        cpu: 1000m
        memory: 2Gi
      resourceDiffPercentage: 20
      controlledResources: ["cpu", "memory"]
    mixcoord:
      trigger: "On"
      podLifeTimeThreshold: 10m
      minAllowed:
        cpu: 100m
        memory: 1Gi
      maxAllowed:
        cpu: 1000m
        memory: 2Gi
      resourceDiffPercentage: 20
      controlledResources: ["cpu", "memory"]
    datanode:
      trigger: "On"
      podLifeTimeThreshold: 10m
      minAllowed:
        cpu: 100m
        memory: 1Gi
      maxAllowed:
        cpu: 1000m
        memory: 2Gi
      resourceDiffPercentage: 20
      controlledResources: ["cpu", "memory"]
    querynode:
      trigger: "On"
      podLifeTimeThreshold: 10m
      minAllowed:
        cpu: 100m
        memory: 1Gi
      maxAllowed:
        cpu: 1000m
        memory: 2Gi
      resourceDiffPercentage: 20
      controlledResources: ["cpu", "memory"]
    streamingnode:
      trigger: "On"
      podLifeTimeThreshold: 10m
      minAllowed:
        cpu: 100m
        memory: 1Gi
      maxAllowed:
        cpu: 1000m
        memory: 2Gi
      resourceDiffPercentage: 20
      controlledResources: ["cpu", "memory"]
  storage:
    streamingnode:
      trigger: "On"
      usageThreshold: 34
      expansionMode: "Online"
      scalingRules:
        - appliesUpto: "100Ti"
          threshold: "50%"
  opsRequestOptions:
    apply: IfReady
    timeout: 10m

MilvusAutoscaler Spec

spec.databaseRef

spec.databaseRef is required. It points to the Milvus object that will be autoscaled.

spec:
  databaseRef:
    name: milvus-cluster

spec.compute

spec.compute configures CPU and memory autoscaling.

The valid keys depend on the Milvus topology:

  • Standalone: node
  • Distributed: proxy, mixcoord, datanode, querynode, streamingnode

Each compute block supports:

  • trigger - On or Off.
  • podLifeTimeThreshold - minimum pod age before autoscaling is considered.
  • minAllowed - lower resource bound.
  • maxAllowed - upper resource bound.
  • resourceDiffPercentage - minimum percentage difference between current resources and the recommendation before a scaling operation is created.
  • controlledResources - which resources may be autoscaled, typically cpu and memory.

For compute autoscaling, the autoscaler’s built-in recommender produces a recommendation first (recorded in status.vpas; no VerticalPodAutoscaler object is created) and then generates a VerticalScaling MilvusOpsRequest when the recommendation passes the configured thresholds.

See Compute Autoscaling Overview.

spec.storage

spec.storage configures storage autoscaling.

The valid keys depend on the Milvus topology:

  • Standalone: node
  • Distributed: streamingnode

Only streamingnode is supported for distributed Milvus because it is the only distributed role with persistent Milvus storage.

Each storage block supports:

  • trigger - On or Off.
  • usageThreshold - disk usage percentage that triggers expansion.
  • expansionMode - Online or Offline.
  • scalingRules - how much storage should be added when usage crosses the threshold.

For storage autoscaling, the autoscaler reads PVC usage from the custom.metrics.k8s.io API served by KubeDB’s storage metrics server and creates a VolumeExpansion MilvusOpsRequest when the configured threshold is crossed.

See Storage Autoscaling Overview.

spec.opsRequestOptions

spec.opsRequestOptions controls how the internally generated MilvusOpsRequest should be created and executed.

The Milvus autoscaling guides currently use:

  • apply - for example IfReady.
  • timeout - the maximum time each generated ops request should get.

Example:

spec:
  opsRequestOptions:
    apply: IfReady
    timeout: 10m

How It Works

Compute autoscaling

  1. A user creates a MilvusAutoscaler with spec.compute.
  2. The autoscaler’s built-in recommender produces one recommendation per targeted component.
  3. When the recommendation differs from current resources by more than resourceDiffPercentage, the autoscaler creates a VerticalScaling MilvusOpsRequest.
  4. The Ops-manager applies the scaling operation.

Storage autoscaling

  1. A user creates a MilvusAutoscaler with spec.storage.
  2. The autoscaler reads persistent-volume usage from the custom.metrics.k8s.io API (KubeDB storage metrics server).
  3. When usage exceeds usageThreshold, the autoscaler creates a VolumeExpansion MilvusOpsRequest.
  4. The Ops-manager applies the volume expansion.

Prerequisites

  • Compute autoscaling requires a metrics server.
  • Storage autoscaling requires KubeDB’s storage metrics server, enabled with --set kubedb-autoscaler.storage-metrics-server.enabled=true in the KubeDB install.
  • Storage autoscaling also requires an expansion-capable StorageClass.

What's on this Page

Search
Improve This Page