Embedding Microservice with TEI

Table of Contents

  1. Start Microservice with docker run

  2. Start Microservice with Docker Compose

  3. Consume Embedding Service

  4. Tips for Better Understanding


Start Microservice with docker run

Start TEI Service

Replace your_port and model with desired values to start the service.

your_port=8090
model="BAAI/bge-large-en-v1.5"
docker run -p $your_port:80 -v ./data:/data --name tei-embedding-serving \
-e http_proxy=$http_proxy -e https_proxy=$https_proxy --pull always \
ghcr.io/huggingface/text-embeddings-inference:cpu-1.6 --model-id $model

Test the TEI service

Run the following command to check if the service is up and running.

curl localhost:$your_port/v1/embeddings \
-X POST \
-d '{"input":"What is Deep Learning?"}' \
-H 'Content-Type: application/json'

Build Docker Image and Run Docker with CLI

  1. Build the Docker image for the embedding microservice:

    cd ../../../
    docker build -t opea/embedding:latest \
    --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy \
    -f comps/embeddings/src/Dockerfile .
    
  2. Run the embedding microservice connected to TEI service:

    docker run -d --name="embedding-tei-server" \
    -p 6000:6000 \
    -e http_proxy=$http_proxy -e https_proxy=$https_proxy \
    --ipc=host \
    -e TEI_EMBEDDING_ENDPOINT=$TEI_EMBEDDING_ENDPOINT \
    -e EMBEDDING_COMPONENT_NAME="OPEA_TEI_EMBEDDING" \
    opea/embedding:latest
    

Start Microservice with Docker Compose

Deploy both the TEI Embedding Service and the Embedding Microservice using Docker Compose

  1. Export environment variables:

    export host_ip=${your_ip_address}
    export EMBEDDING_MODEL_ID="BAAI/bge-base-en-v1.5"
    export TEI_EMBEDDER_PORT=8090
    export EMBEDDER_PORT=6000
    export TEI_EMBEDDING_ENDPOINT="http://${host_ip}:${TEI_EMBEDDER_PORT}"
    
  2. Navigate to the Docker Compose directory:

    cd comps/embeddings/deployment/docker_compose/
    
  3. Start the services:

    docker compose up tei-embedding-serving tei-embedding-server -d
    

Consume Embedding Service

Check Service Status

Verify the embedding service is running:

curl http://localhost:6000/v1/health_check \
-X GET \
-H 'Content-Type: application/json'

Use the Embedding Service API

The API is compatible with the OpenAI API.

  1. Single Text Input

    curl http://localhost:6000/v1/embeddings \
    -X POST \
    -d '{"input":"Hello, world!"}' \
    -H 'Content-Type: application/json'
    
  2. Multiple Text Inputs with Parameters

    curl http://localhost:6000/v1/embeddings \
    -X POST \
    -d '{"input":["Hello, world!","How are you?"], "dimensions":100}' \
    -H 'Content-Type: application/json'
    

Tips for Better Understanding

  1. Port Mapping
    Ensure the ports are correctly mapped to avoid conflicts with other services.

  2. Model Selection
    Choose a model appropriate for your use case, like "BAAI/bge-large-en-v1.5" or "BAAI/bge-base-en-v1.5".

  3. Environment Variables
    Use http_proxy and https_proxy for proxy setup if necessary.

  4. Data Volume
    The -v ./data:/data flag ensures the data directory is correctly mounted.