Deploy TGI on kubernetes cluster

• You should have Helm (version >= 3.15) installed. Refer to the Helm Installation Guide for more information.

• For more deployment options, refer to helm charts README.

Deploy on Xeon

export HFTOKEN="insert-your-huggingface-token-here"
helm install tgi oci://ghcr.io/opea-project/charts/tgi  --set global.HUGGINGFACEHUB_API_TOKEN=${HFTOKEN} -f cpu-values.yaml

Deploy on Gaudi

export HFTOKEN="insert-your-huggingface-token-here"
helm install tgi oci://ghcr.io/opea-project/charts/tgi  --set global.HUGGINGFACEHUB_API_TOKEN=${HFTOKEN} -f gaudi-values.yaml