whisper

Helm chart for deploying whisper service.

Installing the Chart

To install the chart, run the following:

export MODELDIR=/mnt/opea-models
export ASR_MODEL_PATH="openai/whisper-small"
helm install whisper whisper --set global.modelUseHostPath=${MODELDIR} --set ASR_MODEL_PATH=${ASR_MODEL_PATH}

Install the microservice in air gapped (offline) mode

To run whisper microservice in an air gapped environment, users are required to pre-download the model openai/whisper-small to a shared storage.

Use node-local directory

Assuming the model data is shared using node-local directory /mnt/opea-models.

# On every K8s node, run the following command:
export MODEL_DIR=/mnt/opea-models
# Download model, assumes Python huggingface_hub[cli] module is already installed
huggingface-cli download --cache-dir "${MODEL_DIR}" openai/whisper-small

# On K8s master node, run the following command:
# Install using Helm with the following additional parameters:
helm install ... --set global.offline=true,global.modelUseHostPath=${MODEL_DIR}

Use persistent volume

Assuming we share the offline data on cluster level using a persistent volume (PV), first we need to create the persistent volume claim (PVC) with name opea-model-pvc to store the model data.

# Download model openai/whisper-small at the root directory of the corresponding PV
# ...
# Install using Helm with the following additional parameters:
# export MODEL_PVC=opea-model-pvc
# helm install ... --set global.offline=true,global.modelUsePVC=${MODEL_PVC}

Verify

Use port-forward to access it from localhost.

kubectl port-forward service/whisper 1234:7066 &
curl http://localhost:1234/v1/asr \
  -XPOST \
  -d '{"audio": "UklGRigAAABXQVZFZm10IBIAAAABAAEARKwAAIhYAQACABAAAABkYXRhAgAAAAEA"}' \
  -H 'Content-Type: application/json'

Values

Key	Type	Default	Description
image.repository	string	"opea/whisper"
service.port	string	"7066"
global.offline	bool	false	Whether to run the microservice in air gapped environment
global.modelUseHostPath	String	""	Cached models directory on Kubernetes node, service will not download if the model is cached here. The host path “modelUseHostPath” will be mounted to the container as /data directory. Setting this to null/empty will force the pod to download the model every time during startup. May not be set if global.modelUsePVC is also set.
global.modelUsePVC	String	""	Name of Persistent Volume Claim to use for model cache. The Persistent Volume will be mounted to the container as /data directory. Setting this to null/empty will force the pod to download the model every time during startup. May not be set if global.modelUseHostPath is also set.
global.HF_TOKEN	string	insert-your-huggingface-token-here	Hugging Face API token