# whisper Helm chart for deploying whisper service. ## Installing the Chart To install the chart, run the following: ```console export MODELDIR=/mnt/opea-models export ASR_MODEL_PATH="openai/whisper-small" helm install whisper whisper --set global.modelUseHostPath=${MODELDIR} --set ASR_MODEL_PATH=${ASR_MODEL_PATH} ``` ### Install the microservice in air gapped (offline) mode To run `whisper` microservice in an air gapped environment, users are required to pre-download the model `openai/whisper-small` to a shared storage. #### Use node-local directory Assuming the model data is shared using node-local directory `/mnt/opea-models`. ``` # On every K8s node, run the following command: export MODEL_DIR=/mnt/opea-models # Download model, assumes Python huggingface_hub[cli] module is already installed huggingface-cli download --cache-dir "${MODEL_DIR}" openai/whisper-small # On K8s master node, run the following command: # Install using Helm with the following additional parameters: helm install ... --set global.offline=true,global.modelUseHostPath=${MODEL_DIR} ``` #### Use persistent volume Assuming we share the offline data on cluster level using a persistent volume (PV), first we need to create the persistent volume claim (PVC) with name `opea-model-pvc` to store the model data. ``` # Download model openai/whisper-small at the root directory of the corresponding PV # ... # Install using Helm with the following additional parameters: # export MODEL_PVC=opea-model-pvc # helm install ... --set global.offline=true,global.modelUsePVC=${MODEL_PVC} ``` ## Verify Use port-forward to access it from localhost. ```console kubectl port-forward service/whisper 1234:7066 & curl http://localhost:1234/v1/asr \ -XPOST \ -d '{"audio": "UklGRigAAABXQVZFZm10IBIAAAABAAEARKwAAIhYAQACABAAAABkYXRhAgAAAAEA"}' \ -H 'Content-Type: application/json' ``` ## Values | Key | Type | Default | Description | | ----------------------- | ------ | ------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | image.repository | string | `"opea/whisper"` | | | service.port | string | `"7066"` | | | global.offline | bool | `false` | Whether to run the microservice in air gapped environment | | global.modelUseHostPath | String | `""` | Cached models directory on Kubernetes node, service will not download if the model is cached here. The host path "modelUseHostPath" will be mounted to the container as /data directory. Setting this to null/empty will force the pod to download the model every time during startup. May not be set if `global.modelUsePVC` is also set. | | global.modelUsePVC | String | `""` | Name of Persistent Volume Claim to use for model cache. The Persistent Volume will be mounted to the container as /data directory. Setting this to null/empty will force the pod to download the model every time during startup. May not be set if `global.modelUseHostPath` is also set. | | global.HF_TOKEN | string | `insert-your-huggingface-token-here` | Hugging Face API token |