Prompt Template microservice¶

The Prompt Template microservice dynamically generates system and user prompts based on structured inputs and document context. It supports usage in LLM pipelines to customize prompt formatting with reranked documents, conversation history, and user queries.

Getting started¶

🚀1. Start Prompt Template Microservice with Python (Option 1)¶

To start the Prompt Template microservice, you need to install Python packages first.

1.1. Install Requirements¶

pip install -r requirements.txt

1.2. Start Microservice¶

python opea_prompt_template_microservice.py

🚀2. Start Prompt Template Microservice with Docker (Option 2)¶

2.1. Build the Docker Image:¶

Use the below docker build command to create the image:

cd ../../../
docker build -t opea/prompt-template:latest -f comps/prompt_template/src/Dockerfile .

Please note that the building process may take a while to complete.

2.2. Run the Docker Container:¶

docker run -d --name="prompt-template-microservice" \
  -p 7900:7900 \
  --net=host \
  --ipc=host \
  opea/prompt-template:latest

3. Verify the Prompt Template Microservice¶

3.1. Check Status¶

curl http://localhost:7900/v1/health_check \
  -X GET \
  -H 'Content-Type: application/json'

3.2. Sending a Request¶

3.2.1 Default Template Generation¶

Generates the prompt using the default template:

Example Input

curl -X POST -H "Content-Type: application/json" -d @- http://localhost:7900/v1/prompt_template <<JSON_DATA
{
  "data": {
    "user_prompt": "What is Deep Learning?",
    "reranked_docs": [{ "text": "Deep Learning is a subfield of machine learning..." }]
  },
  "conversation_history": [
    { "question": "Hello", "answer": "Hello as well" },
    { "question": "How are you?", "answer": "I am good, thank you!" }
  ],
  "system_prompt_template": "",
  "user_prompt_template": ""
}
JSON_DATA

Example Output

A chat_template starting with the default assistant description.

{
  "id": "4e799abdf5f09433adc276b511a8b0ae",
  "model": null,
  "query": "What is Deep Learning?",
  "max_tokens": 1024,
  "max_new_tokens": 1024,
  "top_k": 10,
  "top_p": 0.95,
  "typical_p": 0.95,
  "temperature": 0.01,
  "frequency_penalty": 0.0,
  "presence_penalty": 0.0,
  "repetition_penalty": 1.03,
  "stream": true,
  "language": "auto",
  "input_guardrail_params": null,
  "output_guardrail_params": null,
  "chat_template": "### You are a helpful, respectful, and honest assistant to help the user with questions. Please refer to the search results obtained from the local knowledge base. Refer also to the conversation history if you think it is relevant to the current question. Ignore all information that you think is not relevant to the question. If you don'\''t know the answer to a question, please don'\''t share false information. \n ### Search results: [File: Unknown Source]\nDeep Learning is...\n### Conversation history: User: Hello\nAssistant: Hello as well\nUser: How are you?\nAssistant: I am good, thank you!\nUser: Who are you?\nAssistant: I am a robot\n### Question: What is Deep Learning? \n\n### Answer:",
  "documents": []
}

3.2.2 Custom Prompt Template¶

You can provide custom system and user prompt templates:

Example Input

curl -X POST -H "Content-Type: application/json" -d @- http://localhost:7900/v1/prompt_template <<JSON_DATA
{
  "data": {
    "initial_query": "What is Deep Learning?",
    "reranked_docs": [{ "text": "Deep Learning is..." }]
  },
  "system_prompt_template": "### Please refer to the search results obtained from the local knowledge base. But be careful to not incorporate information that you think is not relevant to the question. If you don't know the answer to a question, please don't share false information. ### Search results: {reranked_docs}",
  "user_prompt_template": "### Question: {initial_query} \\n### Answer:"
}
JSON_DATA

Example Output

Custom instructions about using search results in the chat_template.

{
  "id": "b1f1cec396954d5dc1b942f5959d556d",
  "model": null,
  "query": "What is Deep Learning?",
  "max_tokens": 1024,
  "max_new_tokens": 1024,
  "top_k": 10,
  "top_p": 0.95,
  "typical_p": 0.95,
  "temperature": 0.01,
  "frequency_penalty": 0.0,
  "presence_penalty": 0.0,
  "repetition_penalty": 1.03,
  "stream": true,
  "language": "auto",
  "input_guardrail_params": null,
  "output_guardrail_params": null,
  "chat_template": "### Please refer to the search results obtained from the local knowledge base. But be careful to not incorporate information that you think is not relevant to the question. If you don'\''t know the answer to a question, please don'\''t share false information. ### Search results: [File: Unknown Source]\nDeep Learning is...\n### Question: What is Deep Learning? \n### Answer:",
  "documents": []
}

3.2.3 Translation Scenario¶

Using a translation-related prompt template:

Example Input

curl -X POST -H "Content-Type: application/json" -d @- http://localhost:7900/v1/prompt_template <<JSON_DATA
{
  "data": {
    "initial_query": "What is Deep Learning?",
    "source_lang": "chinese",
    "target_lang": "english"
  },
  "system_prompt_template": "### You are a helpful, respectful, and honest assistant to help the user with translations. Translate this from {source_lang} to {target_lang}.",
  "user_prompt_template": "### Question: {initial_query} \\n### Answer:"
}
JSON_DATA

Example Output

A translation instruction like: Translate this from chinese to english.

{
  "id": "4f5e0024c2330a7be065b370d02e061f",
  "model": null,
  "query": "什么是深度学习？",
  "max_tokens": 1024,
  "max_new_tokens": 1024,
  "top_k": 10,
  "top_p": 0.95,
  "typical_p": 0.95,
  "temperature": 0.01,
  "frequency_penalty": 0.0,
  "presence_penalty": 0.0,
  "repetition_penalty": 1.03,
  "stream": true,
  "language": "auto",
  "input_guardrail_params": null,
  "output_guardrail_params": null,
  "chat_template": "### You are a helpful, respectful, and honest assistant to help the user with translations. Translate this from chinese to english.\n### Question: 什么是深度学习？ \n### Answer:",
  "documents": []
}