whisper

Helm chart for deploying whisper service.

Installing the Chart

To install the chart, run the following:

export MODELDIR=/mnt/opea-models
export ASR_MODEL_PATH="openai/whisper-small"
helm install whisper whisper --set global.modelUseHostPath=${MODELDIR} --set ASR_MODEL_PATH=${ASR_MODEL_PATH}

Install the microservice in air gapped (offline) mode

To run whisper microservice in an air gapped environment, users are required to pre-download the model openai/whisper-small to a shared storage.

Use node-local directory

Assuming the model data is shared using node-local directory /mnt/opea-models.

# On every K8s node, run the following command:
export MODEL_DIR=/mnt/opea-models
# Download model, assumes Python huggingface_hub[cli] module is already installed
huggingface-cli download --cache-dir "${MODEL_DIR}" openai/whisper-small

# On K8s master node, run the following command:
# Install using Helm with the following additional parameters:
helm install ... --set global.offline=true,global.modelUseHostPath=${MODEL_DIR}

Use persistent volume

Assuming we share the offline data on cluster level using a persistent volume (PV), first we need to create the persistent volume claim (PVC) with name opea-model-pvc to store the model data.

# Download model openai/whisper-small at the root directory of the corresponding PV
# ...
# Install using Helm with the following additional parameters:
# export MODEL_PVC=opea-model-pvc
# helm install ... --set global.offline=true,global.modelUsePVC=${MODEL_PVC}

Verify

Use port-forward to access it from localhost.

kubectl port-forward service/whisper 1234:7066 &
curl http://localhost:1234/v1/asr \
  -XPOST \
  -d '{"audio": "UklGRigAAABXQVZFZm10IBIAAAABAAEARKwAAIhYAQACABAAAABkYXRhAgAAAAEA"}' \
  -H 'Content-Type: application/json'

Values

Key

Type

Default

Description

image.repository

string

"opea/whisper"

service.port

string

"7066"

global.offline

bool

false

Whether to run the microservice in air gapped environment

global.modelUseHostPath

String

""

Cached models directory on Kubernetes node, service will not download if the model is cached here. The host path “modelUseHostPath” will be mounted to the container as /data directory. Setting this to null/empty will force the pod to download the model every time during startup. May not be set if global.modelUsePVC is also set.

global.modelUsePVC

String

""

Name of Persistent Volume Claim to use for model cache. The Persistent Volume will be mounted to the container as /data directory. Setting this to null/empty will force the pod to download the model every time during startup. May not be set if global.modelUseHostPath is also set.

global.HF_TOKEN

string

insert-your-huggingface-token-here

Hugging Face API token