embedding-usvc¶

Helm chart for deploying OPEA embedding microservice.

Installing the chart¶

The OPEA embedding microservice depends on one of the following backend services:

TEI: please refer to tei chart for more information
multimodal embedding BridgeTower: please refer to mm-embedding chart for more information.
prediction guard: please refert to external Prediction Guard for more information.

First, you need to get the dependent service deployed, i.e. deploy the tei helm chart, mm-embedding helm chart, or contact prediction guard to get access info.

After you’ve deployed the dependent service successfully, please run kubectl get svc to get the backend service URL, e.g. http://tei, http://mm-embedding.

To install the embedding-usvc chart, run the following:

cd GenAIInfra/helm-charts/common/embedding-usvc
helm dependency update

# Use TEI as the backend(default)
export EMBEDDING_BACKEND="TEI"
export EMBEDDING_ENDPOINT="http://tei"
helm install embedding-usvc . --set EMBEDDING_BACKEND=${EMBEDDING_BACKEND} --set EMBEDDING_ENDPOINT=${EMBEDDING_ENDPOINT}

# Use multimodal embedding BridgeTower as the backend
# export EMBEDDING_BACKEND="BridgeTower"
# export EMBEDDING_ENDPOINT="http://mm-embedding"
# helm install embedding-usvc . --set EMBEDDING_BACKEND=${EMBEDDING_BACKEND} --set EMBEDDING_ENDPOINT=${EMBEDDING_ENDPOINT}

# Use predcition guard as the backend
# export EMBEDDING_BACKEND="PredictionGuard"
# export API_KEY=<your PedictionGuard api key>
# helm install embedding-usvc . --set EMBEDDING_BACKEND=${EMBEDDING_BACKEND} --set PREDICTIONGUARD_API_KEY=${API_KEY}

Verify¶

To verify the installation, run the command kubectl get pod to make sure all pods are running.

Then run the command kubectl port-forward svc/embedding-usvc 6000:6000 to expose the embedding-usvc service for access.

Open another terminal and run the following command to verify the service if working:

# Verify with TEI or prediction guard backend:
curl http://localhost:6000/v1/embeddings \
    -X POST \
    -H 'Content-Type: application/json' \
    -d '{"input":"What is Deep Learning?"}'

# Verify with multimodal embedding BridgeTower backend:
curl http://localhost:6000/v1/embeddings \
    -X POST \
    -H 'Content-Type: application/json' \
    -d '{"text": {"text" : "This is some sample text."}, "image" : {"url": "https://github.com/docarray/docarray/blob/main/tests/toydata/image-data/apple.png?raw=true"}}'

Values¶

Key	Type	Default	Description
service.port	string	`"6000"`
EMBEDDING_BACKEND	string	`"TEI"`	backend engine to use, one of “TEI”, “BridgeTower”, “PredictionGuard”
EMBEDDING_ENDPOINT	string	`""`
global.monitoring	bool	`false`