OPEA™
latest
OPEA Project
v: latest
Document Versions
latest
1.0
1.1
1.2
1.3
OPEA Project links
Project Home
Wiki
Documentation Home
OPEA Overview
OPEA Project Architecture
Microservices: Flexible and Scalable Architecture
Megaservices: A Comprehensive Solution
Gateways: Customized Access to Mega- and Microservices
Next Step
Open Platform for Enterprise AI (OPEA) Framework Draft Proposal
1. Summary
2. Introduction
3. Framework Components, Architecture and Flow
4. Assessing GenAI components and flows
5. Grading Structure
6. Reference flows
Appendix A – Draft OPEA Specifications
Getting Started with OPEA
Understanding OPEA’s Core Components
Prerequisites
Create and Configure a Virtual Server
Deploy the ChatQnA Solution
Interact with ChatQnA
What’s Next
Get Involved
OPEA Tutorials
AgentQnA
Overview
Purpose
How It Works
Deployment
Single Node
AudioQnA
Overview
Purpose
Key Implementation Details
How It Works
Deployment
Single Node
ChatQnA
Overview
Purpose
Key Implementation Details
How It Works
Customize with new VectorDB
Expected Output
Validation Matrix and Prerequisites
Architecture
Microservice Outline and Diagram
Deployment
Single Node
Xeon Scalable Processor
Gaudi AI Accelerator
Nvidia GPU
AI PC
Kubernetes
Getting Started
Kubernetes Deployment with Helm on Xeon
Cloud Native
Troubleshooting
Monitoring
Set Up the Prometheus Server
Set Up the Grafana Dashboard
Summary and Next Steps
CodeGen
Overview
Purpose
How It Works
Deployment
Intel® Xeon® Scalable processor
Gaudi AI Accelerator
Code Translation
Overview
Purpose
How It Works
Deployment
Single Node
DocSum
Overview
Purpose
How It Works
Deployment
Intel® Xeon® Scalable processor
Gaudi AI Accelerator
DocIndexRetriever
Overview
Purpose
Key Implementation Details
How It Works
Deployment
Single Node
VideoQnA
Overview
Purpose
How It Works
Deployment
Enterprise Inference Guide
Overview
How It Works
Setting Up a Remote Server or Cluster
Using Remote Endpoints on OPEA GenAIExamples
1. Endpoints with Megaservices
2. Endpoints with Microservices
Next Steps
OpenTelemetry on OPEA Guide
Overview
How It Works
How to Monitor
1. Prometheus
2. Grafana
3. Jaeger
Code Instrumentations for OPEA Tracing
OpenTelemetry on GenAIExamples
ChatQnA
AgentQnA
GenAI Examples
Generative AI Examples
Introduction
Architecture
Use Cases
Documentation
Getting Started
Deployment Guide
Supported Examples
Validated Configurations
Contributing to OPEA
Additional Content
Examples
AgentQnA Application
Agents for Question and Answering Application
Agents for Question and Answering Application
Build Service Docker Image
Deploy AgentQnA on AMD GPU (ROCm)
Deploying AgentQnA on Intel® Xeon® Processors
Deploying AgentQnA on Intel® Gaudi® Processors
Deploy AgentQnA on Kubernetes cluster
Retrieval tool for agent
AudioQnA Application
AudioQnA Application
AudioQnA Docker Image Build
AudioQnA Accuracy
AudioQnA Benchmarking
Deploying AudioQnA on AMD EPYC™ Processors
Deploying AudioQnA on AMD ROCm GPU
Deploying AudioQnA on Intel® Xeon® Processors
Deploy AudioQnA application
Deploying AudioQnA on Intel® Gaudi® Processors
Deploy AudioQnA in Kubernetes Cluster on Xeon and Gaudi
Deploy AudioQnA on Kubernetes cluster
AudioQnA E2E test scripts
AudioQnA
AvatarChatbot Application
AvatarChatbot Application
Build Mega Service of AvatarChatbot on AMD GPU
Example AvatarChatbot Deployment on Intel® Xeon® Platform
Example AvatarChatbot Deployment on Intel® Gaudi® Platform
AvatarChatbot E2E test scripts
ChatQnA Application
ChatQnA Application
ChatQnA Docker Image Build
ChatQnA Benchmark Results
ChatQnA Accuracy
FaqGen Accuracy
FaqGen Benchmarking
Deploying ChatQnA on AMD EPYC™ Processors
Deploying FAQ Generation on AMD EPYC™ Processors
Deploying ChatQnA with Pinecone on AMD EPYC™ Processors
Deploying ChatQnA with Qdrant on AMD EPYC™ Processors
Deploying ChatQnA on AMD ROCm GPU
Build Mega Service of ChatQnA on AIPC
Deploying ChatQnA on Intel® Xeon® Processors
Deploying FAQ Generation on Intel® Xeon® Processors
Deploying ChatQnA with MariaDB Vector on Intel® Xeon® Processors
Deploying ChatQnA with Pinecone on Intel® Xeon® Processors
Deploying ChatQnA with Qdrant on Intel® Xeon® Processors
Example ChatQnA deployments on an Intel® Gaudi® Platform
How to Check and Validate Micro Service in the GenAI Example
Build MegaService of ChatQnA on NVIDIA GPU
Deploy ChatQnA in Kubernetes Cluster on Xeon and Gaudi
Deploy ChatQnA on Kubernetes cluster
ChatQnA E2E test scripts
ChatQnA Conversational UI
ChatQnA Customized UI
CodeGen Application
Code Generation Example (CodeGen)
CodeGen Accuracy Benchmark
CodeGen Performance Benchmark
Deploy CodeGen Application on AMD EPYC™ Processors with Docker Compose
Deploy CodeGen Application on AMD GPU (ROCm) with Docker Compose
To deploy the CodeGen services, execute the
docker
compose
up
command with the appropriate arguments. For a TGI deployment, execute:
Deploy CodeGen Application on Intel Xeon CPU with Docker Compose
Deploy CodeGen Application on Intel Gaudi HPU with Docker Compose
Deploy CodeGen using Kubernetes Microservices Connector (GMC)
Deploy CodeGen on Kubernetes using Helm
CodeGen E2E test scripts
Document Summary
Code Gen
Code Gen
CodeTrans Application
Code Translation Application
CodeTrans Docker Image Build
CodeTrans Benchmarking
Deploy CodeTrans Application on AMD EPYC™ Processors with Docker Compose
Deploying CodeTrans on AMD ROCm GPU
Deploying CodeTrans on Intel® Xeon® Processors
Deploying CodeTrans on Intel® Gaudi® Processors
Deploy CodeTrans in a Kubernetes Cluster
Deploy CodeTrans on Kubernetes cluster
CodeTrans E2E test scripts
Code Translation
DBQnA Application
DBQnA Application
Example DBQnA Deployment on AMD GPU (ROCm)
Example DBQnA Deployment on Intel® Xeon® Platform
DBQnA E2E test scripts
DBQnA React Application
DeepResearchAgent Application
Deep Research Agent Application
DocIndexRetriever Application
DocRetriever Application
DocRetriever Application with Docker
DocRetriever Application with Docker
DocIndexRetriever E2E test scripts
DocSum Application
Document Summarization Application
Table of Contents
Deploy DocSum Application on AMD EPYC™ Processors with Docker Compose
Example DocSum deployments on AMD GPU (ROCm)
Example DocSum deployments on Intel Xeon Processor
Example DocSum deployments on Intel® Gaudi® Platform
Deploy DocSum in Kubernetes Cluster
Deploy DocSum on Kubernetes cluster
DocSum E2E test scripts
Document Summary
Doc Summary React
Doc Summary
EdgeCraftRAG Application
Edge Craft Retrieval-Augmented Generation Application
Example Edge Craft Retrieval-Augmented Generation Deployment on Intel® Arc® Platform
Edge Craft Retrieval-Augmented Generation API guide
Edge Craft Retrieval-Augmented Generation Advanced Setup
Explore Edge Craft RAG
Query Search
快速启动指南
EdgeCraftRAG E2E test scripts
EdgeCraftRAG one-click deployment script
FinanceAgent Application
Finance Agent Example
Deploy Finance Agent on Intel® Gaudi® AI Accelerator with Docker Compose
GraphRAG Application
GraphRAG Application
About GraphRAG LLMs
GraphRAG Application
Example GraphRAG Deployment on Intel® Gaudi® Platform
GraphRAG Example Datasets
GraphRAG E2E test scripts
ChatQnA Conversational UI
ChatQnA Customized UI
HybridRAG Application
HybridRAG Application
Notice for FFmpeg:
Example HybridRAG deployments on an Intel® Gaudi® Platform
ChatQnA Customized UI
InstructionTuning Application
Instruction Tuning
Table of contents
Deploy Instruction Tuning Service on Intel® Xeon® Processors
Table of contents
Deploy Instruction Tuning Service on Intel® Gaudi® Platform
Table of contents
Translation E2E test scripts
MultimodalQnA Application
MultimodalQnA Application
MultimodalQnA Docker Image Build
Build Mega Service of MultimodalQnA on AMD EPYC™ Processors
Deploying MultimodalQnA on AMD GPU (ROCm)
Table of Contents
Deploying MultimodalQnA on Intel® Xeon® Processors
Deploying MultimodalQnA on Intel® Gaudi® Processors
MultimodalQnA E2E test scripts
ProductivitySuite Application
Productivity Suite Application
Build Mega Service of Productivity Suite on AMD EPYC™ Processors
🔐 Keycloak Configuration Setup
Example Productivity Suite Deployment on Intel® Xeon® Platform
🔐 Keycloak Configuration Setup
ProductivitySuite E2E test scripts
Productivity Suite React UI
RerankFinetuning Application
Rerank Model Finetuning
Deploy Rerank Model Finetuning Service on Xeon
Deploy Rerank Model Finetuning Service on Gaudi
SearchQnA Application
SearchQnA Application
SearchQnA Docker Image Build
Deploying SearchQnA on AMD EPYC™ Processors
Deploying SearchQnA on AMD ROCm Platform
Deploying SearchQnA on Intel® Xeon® Processors
Deploying searchqna on Intel® Gaudi® Processors
Deploy SearchQnA in a Kubernetes Cluster
Deploy SearchQnA on Kubernetes cluster
SearchQnA E2E test scripts
Neural Chat
Text2Image Application
Text-to-Image Microservice
Deploy Text-to-Image Service on AMD EPYC™ Processors
Deploy Text-to-Image Service on Xeon
Deploy Text-to-Image Service on Gaudi
Deploy txt2img on Kubernetes cluster
Text2Image E2E test scripts
Text2Image Customized UI
Translation Application
Translation Application
Example Translation Deployment on AMD EPYC™ Platform
Example Translation Deployment on AMD GPU (ROCm)
Example Translation Deployment on Intel® Xeon® Platform
Example Translation Deployment on Intel® Gaudi® Platform
Deploy Translation in a Kubernetes Cluster
Translation E2E test scripts
Language Translation
VideoQnA Application
VideoQnA Application
Build Mega Service of VideoQnA on Xeon
VisualQnA Application
Visual Question and Answering
VisualQnA Docker Image Build
VisualQnA Benchmarking
Build Mega Service of VisualQnA on AMD EPYC™ Processors
Deploy VisualQnA on AMD GPU (ROCm)
Deploying VisualQnA on Intel® Xeon® Processors
Deploying VisualQnA on Intel® Gaudi® Processors
Deploy VisualQnA in a Kubernetes Cluster
Deploy VisualQnA on Kubernetes cluster
VisualQnA E2E test scripts
WorkflowExecAgent Application
Workflow Executor Agent
Deploying WorkflowExecAgent on Intel® Xeon® Processors
Validate Workflow Agent Microservice
one_click_deploy Application
One-Click Deployment for GenAI Examples
Legal Information
License
Citation
Deploy and Benchmark
Purpose
Support Example List
Table of Contents
Prerequisites
Data Preparation
Running Deploy and Benchmark Tests
Running the Tests
Test Modes
Troubleshooting
ChatQnA Release Data
Docker Images
Example images
Microservice images
Supported Examples
ChatQnA
CodeGen
CodeTrans
DocSum
Language Translation
SearchQnA
VisualQnA
VideoQnA
RerankFinetuning
InstructionTuning
DocIndexRetriever
AgentQnA
AudioQnA
MultimodalQnA
ProductivitySuite
DBQnA
Text2Image
AvatarChatbot
Validated Configurations
v1.3 Release Test Config Overview
GenAI Microservices
Generative AI Components (GenAIComps)
GenAIComps
Installation
MicroService
MegaService
Contributing to OPEA
Additional Content
Legal Information
License
Citation
Agent Microservice
Deploy Agent microservice on Kubernetes cluster
Deploy on Kubernetes
Agent Microservice
Table of Contents
Quick Start Deployment
Supported agent types
Docker Compose Files
Other Supported Features
Customizations
Plan Execute Agent Microservice
RAG Agent Microservice
SQL Agent Microservice
Table of Contents
Overview of sql_agent_llama
Overview of sql_agent
Limitations
Animation Microservice
Deploying Avatar Animation Service
🚀 1. Quick Start with Docker Compose
🚀 2. Manual Step-by-Step Deployment (Advanced)
Avatar Animation Microservice
Table of contents
Architecture
Deployment Options
Validated Configurations
Asr Microservice
Deploy ASR microservice on Kubernetes cluster
Deploy on Kubernetes
ASR Microservice
🚀1. Start Microservice with Python (Option 1)
🚀2. Start Microservice with Docker (Option 2)
🚀3. Start Microservice with Docker Compose (Option 3)
Chathistory Microservice
📝 Chat History Microservice
🛠️ Features
⚙️ Implementation
Deploy chathistory microservice on Kubernetes cluster
Deploy on Kubernetes
📝 Chat History Microservice with MongoDB
Setup Environment Variables
🚀 Start Microservice with Docker (Option 1)
🚀 Start Microservice with Docker Compose (Option 2)
✅ Invoke Microservice
Cores Microservice
OPEA MCP Tool
OPEA MCP Tool Overview
API Usage
Examples
Contributing to Storage
Storage
Adding a New Storage Backend
Example
Telemetry for OPEA
Metrics
Statistics
Tracing
Visualization
Dataprep Microservice
Dataprep Microservice
Table of contents
Install Requirements
Summarizing Image Data with LVM
Dataprep Microservice on Various Databases
Running in the air gapped environment
Deploy dataprep microservice on Kubernetes cluster
Table of contents
Deploy on Kubernetes with redis VectorDB
Deploy on Kubernetes with milvus VectorDB
Dataprep Microservice with ArangoDB
Table of contents
🚀Start Microservice with Docker
🚀Consume Dataprep Service
Dataprep Microservice with Elasticsearch
Table of contents
🚀1. Start Microservice with Docker
🚀2. Consume Microservice
Dataprep microservice for financial domain data
Table of contents
1. Overview
2. Deploy with docker
3. Consume Microservice
Dataprep Microservice with MariaDB Vector
Table of contents
🚀1. Start Microservice with Docker
🚀2. Consume Microservice
Dataprep Microservice with Milvus
Table of contents
🚀1. Start Microservice with Docker
🚀2. Consume Microservice
🚀3. Troubleshooting
Running in the air gapped environment
Dataprep Microservice for Multimodal Data with Redis
Table of contents
🚀1. Start Microservice with Docker
🚀2. Status Microservice
🚀3. Consume Microservice
Dataprep Microservice with Neo4J
Table of contents
Start Microservice with Docker
Invoke Microservice
Dataprep Microservice with Neo4J
Table of contents
Start Microservice with Docker
Invoke Microservice
Dataprep Microservice with OpenSearch
Table of contents
🚀1. Start Microservice with Docker
🚀2. Status Microservice
🚀3. Consume Microservice
Dataprep Microservice with PGVector
Table of contents
🚀1. Start Microservice with Docker
🚀2. Consume Microservice
Dataprep Microservice with Pinecone
Table of contents
🚀Start Microservice with Docker
Invoke Microservice
Dataprep Microservice with Qdrant
Table of contents
🚀Start Microservice with Docker
Invoke Microservice
Running in the air gapped environment
Dataprep Microservice with Redis
Table of contents
🚀1. Start Microservice with Docker
🚀2. Status Microservice
🚀3. Consume Microservice
Running in the air gapped environment
Dataprep Microservice with VDMS
Table of contents
🚀1. Start Microservice with Docker (Option 2)
🚀2. Status Microservice
🚀3. Consume Microservice
Embeddings Microservice
Embeddings Microservice
Table of Contents
Overview
Key Features
Supported Implementations
Deploy Embedding microservice on Kubernetes cluster
Deploy on Kubernetes
Multimodal Embeddings Microservice
Table of Contents
Start Microservice
Consume Embedding Service
Running in the air gapped environment
Multimodal CLIP Embedding Microservice
Table of Contents
Start Microservice
Consume Embedding Service
Embedding Microservice with OpenVINO Model Server (OVMS)
Table of Contents
Start Microservice with
docker
run
Start Microservice with Docker Compose
Consume Embedding Service
Tips for Better Understanding
Embedding Microservice with Prediction Guard
Table of Contents
Start Microservice with
docker
run
Start Microservice with Docker Compose
Consume Embedding Service
Tips for Better Understanding
Embedding Microservice with TEI
Table of Contents
Start Microservice with
docker
run
Start Microservice with Docker Compose
Consume Embedding Service
Tips for Better Understanding
Feedback_management Microservice
🗨 Feedback Management Microservice
🛠️ Features
⚙️ Implementation
🗨 Feedback Management Microservice with MongoDB
Setup Environment Variables
🚀 Start Microservice with Docker (Option 1)
🚀 Start Microservice with Docker Compose (Option 2)
Finetuning Microservice
Fine-tuning Microservice
Table of contents
🚀1. Start Microservice with Python (Option 1)
🚀2. Start Microservice with Docker (Option 2)
🚀3. Consume Finetuning Service
🚀4. Descriptions for Finetuning parameters
XTune - Model finetune tool for Intel GPU
Table of contents
Installation
Data Preparation
Fine-Tuning with LLaMA Board GUI (powered by Gradio)
Fine-Tuning with Shell instead of GUI
Xtune
Examples
Citation
Acknowledgement
Dataset
Dataset for CLIP
Dataset for AdaCLIP
Preprocessing
Dataset for Qwen2-VL Finetune
Update dataset_info.json
Qwen2-VL Training and Hyperparameter Optimization
🚀 Features
📋 Running Modes
🛠️ Usage
⚙️ Configuration Parameters
🔍 Optuna features
💡 Log Control
📊 Output and Results
📝 Complete Examples
Finetune qwen2-vl with logging eval loss
AdaCLIP-Finetune
Prerequisites
How to Install
Install on NVIDIA
Install on Arc A770
Prepare Datasets
Datasets
Frame Extraction
Dataset JSON prepare
Implemented finetune methods
BitFit & SSF
Importance Based Selection (IBS)
Performance of different finetune methods
How to Finetune
Finetune on NVIDIA
Finetune on Arc A770
Use optuna to automatic get the best param
Visualization
Install linux kernel
Install Driver
Optional:
The core features for clip finetune tool:
How to config features for clip finetune tool:
Basic yaml for vit_b16.yaml
Angle-Based Selection for full-finetune
customize trained layer for partial-finetune
fixed cache size for tip-adapter
Guardrails Microservice
Trust and Safety with LLM
Deploy guardrails microservice on Kubernetes cluster
Deploy on Kubernetes
Bias Detection Microservice
Introduction
Future Development
🚀1. Start Microservice with Python（Option 1）
🚀2. Start Microservice with Docker (Option 2)
🚀3. Get Status of Microservice
🚀4. Consume Microservice Pre-LLM/Post-LLM
Factuality Check Prediction Guard Microservice
🚀 Start Microservice with Docker
Setup Environment Variables
Build Docker Images
Start Service
🚀 Consume Factuality Check Service
Guardrails Microservice
Llama Guard
WildGuard
Clone OPEA GenAIComps and set initial environment variables
Start up the HuggingFace Text Generation Inference (TGI) Server
Hallucination Detection Microservice
Introduction
🚀1. Start Microservice based on vLLM endpoint on Intel Gaudi Accelerator
2. Set up Hallucination Microservice
🚀3. Get Status of Hallucination Microservice
🚀4. Consume Guardrail Micorservice Post-LLM
PII Detection Microservice
Introduction
Environment Setup
🚀1. Start Microservice with Python（Option 1）
🚀2. Start Microservice with Docker (Option 2)
🚀3. Get Status of Microservice
🚀4. Consume Microservice
Prompt Injection and Jailbreak Detection Microservice
Introduction
Prompt Guard Microservice
Prompt Injection Detection Prediction Guard Microservice
Environment Setup
Setup Environment Variables
🚀1. Start Microservice with Docker
Toxicity Detection Microservice
Introduction
Environment Setup
🚀1. Start Microservice with Python（Option 1）
🚀2. Start Microservice with Docker (Option 2)
🚀3. Get Status of Microservice
🚀4. Consume Microservice Pre-LLM/Post-LLM
Image2image Microservice
Image-to-Image Microservice
🚀1. Start Microservice with Python (Option 1)
1.1 Install Requirements
1.2 Start Image-to-Image Microservice
🚀2. Start Microservice with Docker (Option 2)
2.1 Build Images
2.2 Start Image-to-Image Service with Docker
🚀3. Start Image-to-Image with Docker Compose
4 Test Image-to-Image Service
Image2video Microservice
Image-to-Video Microservice
🚀1. Start Microservice with Python (Option 1)
1.1 Install Requirements
1.2 Start Image-to-Video Microservice
🚀2. Start Microservice with Docker (Option 2)
2.1 Build Images
2.2 Start Image-to-Video Service
2.3 Test
Language_detection Microservice
Language Detection microservice
Configuration Options
Getting started
Llms Microservice
Deploy LLM microservice on Kubernetes cluster
Deploy on Kubernetes
Document Summary LLM Microservice
Table of Contents
Start Microservice
Consume Microservice
Air-Gapped Environment Usage
FAQGen LLM Microservice
Table of Contents
Start Microservice
Consume Microservice
LLM Text Generation Microservice
Table of Contents
Validated LLM Models
Start Microservice
Consume Microservice
OPEA Text Generation Microservice (Amazon Bedrock)
Table of Contents
Setup Environment Variables
Build Docker Image
Start Microservice
Consume Microservice
OPEA Text Generation Microservice (OpenAI-Compatible Endpoint)
Table of Contents
Prepare Docker Image
Setup Environment Variables
Start Microservice
Consume Microservice
LLM Native Microservice
Table of Contents
Start Microservice
Consume Microservice
LLM OVMS Microservice
Table of Contents
Start OVMS Microservice
Start OPEA LLM Microservice
Consume Microservice
Tips
Prediction Guard Microservice
Table of Contents
Start Microservice
Consume Microservice
LM-Eval Microservice
CPU service
Lvms Microservice
LVM Microservice
Table of Contents
Overview
Key Features
Supported Implementations
Deploy LVM microservice on Kubernetes cluster
Deploy on Kubernetes
LVM Microservice with LLaMA-Vision
Table of Contents
Start Microservice
Consume LVM Service
LVM Microservice with LLaVA
Table of Contents
Start Microservice
Consume LVM Service
LVM Microservice with TGI-accelerated LLaVA
Table of Contents
Start Microservice
Consume LVM Service
LVM Microservice with PredictionGuard
Table of Contents
Start Microservice
Consume LVM Service
LVM Microservice with Video-LLaMA
Table of Contents
Start Microservice
Consume LVM Service
LVM Microservice with vLLM on Gaudi
Table of Contents
Prerequisites
Start Microservice
Consume LVM Service
Prompt_registry Microservice
🧾 Prompt Registry Microservice
🛠️ Features
⚙️ Implementation
Deploy prompt microservice on Kubernetes cluster
Deploy on Kubernetes
🧾 Prompt Registry Microservice with MongoDB
Setup Environment Variables
🚀 Start Microservice with Docker (Option 1)
🚀 Start Microservice with Docker Compose (Option 2)
Prompt_template Microservice
Prompt Template microservice
Getting started
Rerankings Microservice
Deploy reranking microservice on Kubernetes cluster
Deploy on Kubernetes
Reranking Microservice
🛠️ Features
⚙️ Implementation
🌟 Reranking Microservice with OVMS
📦 1. Prepare the model in the model repository
📦 2. Start Microservice with Docker
📦 3. Start Microservice with docker compose
📦 4. Consume Reranking Service
✨ Tips for Better Understanding:
🌟 Reranking Microservice with TEI
📦 1. Start Microservice with Docker
📦 2. Start Microservice with docker compose
📦 3. Consume Reranking Service
✨ Tips for Better Understanding:
🌟 Reranking Microservice with VideoQnA
📦 1. Start Microservice with Docker
📦 2. Start Microservice with docker compose
📦 3. Consume Reranking Service
✨ Tips for Better Understanding:
Retrievers Microservice
Retriever Microservice
Retriever Microservice with Redis
Retriever Microservice with Milvus
Retriever Microservice with Qdrant
Retriever Microservice with PGVector
Retriever Microservice with VDMS
Retriever Microservice with ElasticSearch
Retriever Microservice with OpenSearch
Retriever Microservice with neo4j
Retriever Microservice with Pathway
Retriever Microservice with MariaDB Vector
Retriever Microservice with ArangoDB
Deploy retriever microservice on Kubernetes cluster
Deploy on Kubernetes with redis vector DB
Deploy on Kubernetes with milvus vector DB
Retriever Microservice with ArangoDB
🚀Start Microservice with Docker
🚀3. Consume Retriever Service
Retriever Microservice
🚀1. Start Microservice with Python (Option 1)
🚀2. Start Microservice with Docker (Option 2)
🚀3. Consume Retriever Service
Retriever Microservice
🚀1. Start Microservice with Docker
🚀2. Consume Retriever Service
Retriever Microservice with Milvus
🚀Start Microservice with Python
🚀Start Microservice with Docker
🚀3. Consume Retriever Service
Retriever Microservice with Neo4J
🚀Start Microservice with Docker
Invoke Microservice
Retriever Microservice
🚀1. Start Microservice with Python (Option 1)
🚀2. Start Microservice with Docker (Option 2)
🚀3. Consume Retriever Service
Retriever Microservice with Pathway
🚀Start Microservices
Retriever Microservice
🚀1. Start Microservice with Python (Option 1)
🚀2. Start Microservice with Docker (Option 2)
🚀3. Consume Retriever Service
Retriever Microservice with Qdrant
1. 🚀Start Microservice with Python (Option 1)
2. 🚀Start Microservice with Docker (Option 2)
🚀3. Consume Retriever Service
Retriever Microservice
🚀1. Start Microservice with Python (Option 1)
🚀2. Start Microservice with Docker (Option 2)
🚀3. Consume Retriever Service
Retriever Microservice
Visual Data Management System (VDMS)
🚀1. Start Microservice with Python (Option 1)
🚀2. Start Microservice with Docker (Option 2)
🚀3. Consume Retriever Service
Router Microservice
Router Microservice
Build
RouteLLM compatibility patch
API Usage
Configuration Reference
Troubleshooting
Testing
Struct2graph Microservice
Struct2Graph Microservice
Features
Implementation
🚀1. Start Microservice with docker run
🚀2. Start Microservice with docker compose
3. Validate the service using API endpoint
Text2cypher Microservice
Deploy text2cypher microservice using docker-compose
Deploy on Intel Gaudi
🛢 Text-to-Cypher Microservice
🛠️ Features
⚙️ Implementation
Text2graph Microservice
Text to graph triplet extractor
Decoder Models
Encoder-decoder models
Features
Implementation
🚀1. Start Microservice with Docker
Install Requirements
Environment variables : Configure LLM Parameters based on the model selected.
Validation and testing
Text to triplets
Check if services are up
Text2image Microservice
Deploy text2image on kubernetes cluster
Deploy on Xeon
Deploy on Gaudi
Text-to-Image Microservice
🚀1. Start Microservice with Python (Option 1)
1.1 Install Requirements
1.2 Start Text-to-Image Microservice
🚀2. Start Microservice with Docker (Option 2)
2.1 Build Images
2.2 Start Text-to-Image Service
3 Test Text-to-Image Service
Text2kg Microservice
Text to knowledge graph (text2kg) microservice
Decoder-Only Models
Features
🚀 1. Start individual microservices using docker cli (Option 1)
🚀 2. Start text2kg and dependent microservices with docker-compose (Option 2)
3. Check the service using API endpoint
Text2sql Microservice
🛢 Text-to-SQL Microservice
🛠️ Features
⚙️ Implementation
Third_parties Microservice
Start ArangoDB Server
1. Download ArangoDB image
2. Configure the password
3. Run ArangoDB service
Deploy mm-embedding on kubernetes cluster
Deploy on Xeon
Deploy on Gaudi
Multimodal Embeddings Microservice with BridgeTower
🚀1. Start MMEI on Gaudi2 HPU
🚀2. Start MMEI on Xeon CPU
Running in the air gapped environment
🚀3. Access the service
Multimodal Embeddings Microservice with CLIP
🚀1. Start Microservice with Docker
🚀2. Consume Embedding Service
Start Elasticsearch server
1. Download Elasticsearch image
2. Run Elasticsearch service
Deploy gpt-sovits on kubernetes cluster
Deploy on Xeon
GPT-SoVITS Microservice
Build the Image
Start the Service
Test
IPEX Serving microservice
🚀1. Build the Docker Image
🚀2. Start the microservice
🚀3. Access the service
LVM Microservice
🚀 Start Microservice with Docker
LVM Microservice
🚀1. Start Microservice with Python (Option 1)
🚀2. Start Microservice with Docker (Option 2)
Start MariaDB Server
1. Configure the server
2. Run MariaDB Server
Start Milvus server
1. Configuration
2. Run Milvus service
Deploy MongoDB on kubernetes cluster
Deploy on Xeon
Start Neo4J Server
1. Download Neo4J image
2. Configure the username, password and dbname
3. Run Neo4J service
Deploy nginx on kubernetes cluster
Deploy on Xeon
Introduction
Get Started
Start Opensearch server
Prerequisites
Instructions
Troubleshooting
Deploy OVMS on kubernetes cluster
Deploy on Xeon
Start the Pathway Vector DB Server
Configuration
Building and running
Health check the vector store
Start PGVector server
1. Download Pgvector image
2. Configure the username, password and dbname
3. Run Pgvector service
LVM Prediction Guard Microservice
🚀1. Start Microservice with Python
🚀2. Start Microservice with Docker (Option 2)
🚀3. Consume LVM Service
Start PGVector server
1. Download Pgvector image
2. Configure the username, password and dbname
3. Run Pgvector service
Deploy RedisDB on kubernetes cluster
Deploy on Xeon
Start Redis server
1. Download Redis image
2. Run Redis service
SGLang Serving microservice
🚀1. Build the Docker Image
🚀2. Start the microservice
🚀3. Access the service
Deploy speecht5 on kubernetes cluster
Deploy on Xeon
Deploy on Gaudi
Deploy TEI on kubernetes cluster
Deploy on Xeon
Deploy on Gaudi
TGI LLM Microservice
Start TGI with docker compose
Deploy TGI on kubernetes cluster
Deploy on Xeon
Deploy on Gaudi
Start VDMS server
1. Download VDMS image
2. Run VDMS service
LVM Microservice
🚀1. Start Microservice with Docker
✅ 2. Test
♻️ 3. Clean
vLLM Endpoint Service
🚀1. Set up Environment Variables
🚀2. Set up vLLM Service
🚀3. Set up LLM microservice
Deploy vllm on kubernetes cluster
Deploy on Xeon
Deploy on Gaudi
Deploy whisper on kubernetes cluster
Deploy on Xeon
Deploy on Gaudi
Tts Microservice
Deploy tts microservice on Kubernetes cluster
Deploy on Kubernetes
TTS Microservice
1.2 Start SpeechT5 Service/Test
1.3 Start TTS Service/Test
🚀2. Start Microservice with Docker (Option 2)
🚀3. Start Microservice with Docker Compose (Option 3)
Web_retrievers Microservice
Deploy web-retriever microservice on Kubernetes cluster
Deploy on Kubernetes
Web Retriever Microservice
🚀1. Start Microservice with Docker (Option 1)
🚀2. Start Microservice with Docker Compose (Option 2)
🚀3. Consume Web Retriever Service
Deploying GenAI
GenAIInfra
Overview
Prerequisite
Setup Kubernetes cluster
(Optional) To run GenAIInfra on Intel Gaudi product
Usages
Use GenAI Microservices Connector (GMC) to deploy and adjust GenAIExamples
Use helm charts to deploy
Use terraform to deploy on cloud service providers
Additional Content
Development
Prerequisites
Creating new helm chart for OPEA services
Testing
pre-commit testing
Legal Information
License
Citation
Release Branches
1. Create release candidate branch
2. Create images with release tag
3. Test helm charts
4. Test GMC
5. Publish images
Installation Guides
GenAI-microservices-connector(GMC) Installation
GenAI-microservices-connector(GMC)
Install GMC
Use GMC to compose a chatQnA Pipeline
Kubernetes Installation Options
Kubernetes Installation using AWS EKS Cluster
Prerequisites
Create AWS EKS Cluster in AWS Console
Uploading images to an AWS Private Registry
Kubernetes installation demo using kubeadm
Node configuration
Step 0. Clean up the environment
Step 1. Install relevant components
Step 2. Create the k8s cluster
Step 3 (optional) Reset Kubernetes cluster
NOTES
Kubernetes installation using Kubespray
Node preparation
Prerequisites
Step 1. Set up Kubespray and Ansible
Step 2. Build your own inventory
Step 3. Define Kubernetes configuration
Step 4. Deploy Kubernetes
Step 5. Create kubectl configuration
Quick reference
CI/CD ENV Setup
Setup Test Environment for CI/CD
Install Habana Driver (Gaudi Only)
Install Docker
Install Conda
Install K8S
Setup Image Registry for K8S Test
Setup GHA ENV for CI/CD
Setup Action Runner Controller (ARC)
Cloud Service Provider
OPEA applications AWS EKS deployment guide
Prerequisites
Setup
EKS cluster
Persistent Volume Claim
OPEA Applications
OPEA applications Azure AKS deployment guide
Prerequisites
Setup
AKS cluster
Cosmos DB
Persistent Volume Claim
OPEA Applications
OPEA applications GCP GKE deployment guide
Prerequisites
Setup
Update your opea-chatqna.tfvars file
GKE cluster
Persistent Volume Claim
OPEA Applications
Authentication and Authorization
Authentication and authorization
Istio based implementation for cloud native environments
APISIX based implementation for cloud native environments
Authentication and Authorization with APISIX and OIDC based Identity provider (Keycloak)
Start ChatQnA service
Starting and Configuring Keycloak
Update values
Install APISIX
Usage
Uninstall
Leveraging Istio to compose an OPEA Pipeline with authentication and authorization enabled
Prerequisite
Perform authentication and authorization via Bearer JWT tokens and curl
Perform authentication and authorization via oauth2-proxy and OIDC provider and UI
Uninstall Istio
Helm Charts
Helm charts for deploying GenAI Components and Examples
Table of Contents
Helm Charts
Deploy with Helm charts
Helm Charts Options
Deploy the Helm Charts on Intel® Xeon® Processors with Intel® Trust Domain Extensions (Intel® TDX)
Using HPA (autoscaling)
Using Persistent Volume
Using Private Docker Hub
Generate manifests from Helm Charts
CI guidelines for helm charts
Table of Contents
Infra Setup
Add new test case
HorizontalPodAutoscaler (HPA) support
Table of Contents
Introduction
Pre-conditions
Gotchas
Enable HPA
Verify
Scaling metric considerations
Deployment of the Helm Charts on Intel® Xeon® Processors with Intel® Trust Domain Extensions (Intel® TDX)
Technical background
Prerequisites
Getting Started
Observability for OPEA Workloads in Kubernetes
Table of Contents
Introduction
Pre-conditions
Install
Verify
agent
Deploy
Verify
Options
asr
(Option1): Installing the chart separately
(Option2): Installing the chart with dependencies automatically
Verify
Values
chathistory-usvc
(Option1): Installing the chart separately
(Option2): Installing the chart with dependencies automatically
Verify
Values
Dashboards
Installing the Chart
Values
data-prep
Installing the chart
Verify
Values
Milvus support
embedding-usvc
Installing the chart
Verify
Values
gpt-sovits
Install the chart
Verify
Values
guardrails-usvc
Installing the chart
Verify
Values
llm-uservice
Installing the chart
Verify
Values
OPEA lvm-serve microservice
Installing the Chart
Verify
Values
lvm-uservice
Installing the chart
Verify
Values
OPEA mm-embedding microservice
Installing the Chart
Verify
Values
mongodb
Install the Chart
Verify
Values
OPEA nginx microservice
Integration with OPEA Applications
Gateway Mode (Central Router)
Deployment Modes
ollama
Installing the Chart
Verify
Values
prompt-usvc
(Option1): Installing the chart separately
(Option2): Installing the chart with dependencies automatically
Verify
Values
redis-vector-db
Install the Chart
Verify
Values
reranking-usvc
(Option1): Installing the chart separately
(Option2): Installing the chart with dependencies automatically
Verify
Values
retriever-usvc
Installing the chart
Verify
Values
Milvus support
speecht5
Installing the Chart
Verify
Values
tei
Installing the Chart
Verify
Values
teirerank
Installing the Chart
Verify
Values
OPEA text2image microservice
Installing the Chart
Verify
Values
tgi
Installing the Chart
Verify
Values
tts
(Option1): Installing the chart separately
(Option2): Installing the chart with dependencies automatically
Verify
Values
vllm
Installing the Chart
Verify
Values
web-retriever
(Option1): Installing the chart separately
(Option2): Installing the chart with dependencies automatically
Verify
Values
whisper
Installing the Chart
Verify
Values
AgentQnA
Deploy
Verify
AudioQnA
Installing the Chart
Verify
Values
ChatQnA
Installing the Chart
Verify
Values
Troubleshooting
ChatQnA Troubleshooting
a function to get the endpoint of service
define the namespace of service
Update a file to database
get the embedding of input
get the retriever docs
reranking the docs
TGI Q and A
REF
CodeGen
Installing the Chart
Verify
Values
CodeTrans
Installing the Chart
Verify
Values
DocSum
Installing the Chart
Verify
Values
FinanceAgent
Deploy
Verify
SearchQnA
Installing the Chart
Verify
Values
txt2img
Installing the Chart
Verify
Values
VisualQnA
Installing the Chart
Verify
Values
KubeAI Operator
KubeAI for OPEA
Features
Models
Installation
Prerequisites
Install KubeAI
Deploying the Models
Text Generation with Llama-3 on CPU
Text Generation with Llama-3 on Gaudi
Text Embeddings with BGE on CPU
Using the Models
CPU Performance Optimization with NRI
Overview
Installation of Balloons Policy Plugin
Configuration of Balloons Policy Plugin
Observability
Model Files
Benchmarking
Additional Notes
Kubernetes Addons
Deploy Kubernetes add-ons for OPEA
Metrics / visualization add-ons
Pre-conditions
Device metrics for Gaudi HW
Extra metrics for OPEA applications
CPU mmetrics from PCM
Importing dashboards to Grafana
More dashboards
OPEA with Istio
Introduction
Deployment
Create Istio gateway with TLS and virtual service
Enforce mTLS between OPEA pods
Cleanup
memory bandwidth exporter
Setup
More flags about memory bandwidth exporter
Microservices Connector
genai-microservices-connector(GMC)
Description
Architecture
Personas
Getting Started
Troubleshooting GMC Custom Resource（CR）
Usage guide for genai-microservices-connector(GMC)
Use GMC to compose a chatQnA Pipeline
Use GMC to adjust the chatQnA Pipeline
Use GMC to delete the chatQnA Pipeline
Use GMC and Istio to compose an OPEA Pipeline with authentication and authorization enabled
ChatQnA Use Cases in Kubernetes Cluster via GMC
Using prebuilt images
Deploy ChatQnA pipeline
Helm chart for genai-microservices-connector(GMC)
Installing the GMC Helm Chart
Check the installation result
Next step
Uninstall
Pipeline Proxy
OPEA Pipeline Proxy
Features
Build
Deployment
Development
Guardrails
Architecture
Deployment
Scripts
Scripts and tools
NVIDIA GPU Quick-Start Guide
Prerequisite
Usages
FAQ and Troubleshooting
Deploy Autoscaling Ray Cluster with KubeRay in Kubernetes Cluster
Install KubeRay
Start Ray Cluster with Autoscaling
Delete Ray Cluster with Autoscaling
Uninstall KubeRay
Contributing
Development
Helm Starter Chart
NOTE
OPEA
microservice
Installing the Chart
Verify
Values
Evaluating GenAI
GenAIEval
Installation
Evaluation
lm-evaluation-harness
bigcode-evaluation-harness
Kubernetes platform optimization
Benchmark
Features
How to use
Grafana Dashboards
Additional Content
Legal Information
License
Citation
Kubernetes Platform Optimization with Resource Management
Introduction
NRI Plugins
Install
Validate policy status
Configure
Validate CPU affinity and hardware alignment in containers
Remove a policy
NRI topology-aware resource policy
Recommendations
GenAIEval Dockerfiles
Gaudi Requirements
Run GenAIEval on Gaudi
Docker Build
Docker Run
Benchmarking OPEA Examples on Intel® Gaudi® AI Processor and Xeon® Processor
Docker Build
Run a OPEA Example using docker compose
Docker Run
OPEA Benchmark Tool
Features
Table of Contents
Installation
Prerequisites
Usage
Configuration
Test Suite Configuration
Test Cases
Auto-Tuning for ChatQnA: Optimizing Resource Allocation in Kubernetes
Key Features
Usage
Configuration Files
Hardrware_info.json
chatqna_neuralchat_rerank_latest.yaml
Tuning Config Parameters
Output
Auto-Tuning for ChatQnA: Optimizing Accuracy by Tuning Model Related Parameters
Prepare Dataset
Run the Tuning script
Setup Prometheus and Grafana to visualize microservice metrics
1. Setup Prometheus
1.1 Node Metrics (optional)
1.2 Intel® Gaudi® Metrics (optional)
2. Setup Grafana
3. Import Grafana Dashboard
StressCli
stresscli.py
Prerequirements
Installation
Usage
How to benchmark pubmed datasets by send query randomly
1. prepare the pubmed datasets
1.1 get pubmed data
1.2 use script to extract data
1.3 prepare 4 dataset files
1.4 prepare the query list
2. How to use pubmed qlist
locust scripts for OPEA ChatQnA
Configuration file
Basic Usage
HELMET: How to Evaluate Long-context Language Models Effectively and Thoroughly
Quick Links
Release Progress
Setup
Data
Running evaluation
Run on Intel Gaudi
Model-based evaluation
Adding new models
Adding new tasks
Dataset correlation analysis
Others
Contacts
Citation
Benchmarks for agentic applications
TAG-Bench for evaluating SQL agents
Overview of TAG-Bench
Getting started
Launch your SQL agent
Run the benchmark
Benchmark results
CRAG Benchmark for Agent QnA systems
Overview
Getting started
CRAG dataset
Launch Agent QnA system
Run CRAG benchmark
Use LLM-as-judge to grade the answers
Validation of LLM-as-judge
Benchmark results for OPEA RAG Agents
Comparison with GPT-4o-mini
AutoRAG to evaluate the RAG system performance
Service preparation
RAG evaluation
Notes
🚀 QuickStart
Installation
Launch Service of LLM-as-a-Judge
Writing your first test case
Acknowledgements
Model Card Generator
Steps to generate a Model Card
🚀 QuickStart
Installation
Launch a LLM Service
Example 1: TGI
Example 2: OPEA LLM
Predict
Evaluate
Evaluation Methodology
Introduction
Prerequisite
Environment
MultiHop (English dataset)
Launch Service of RAG System
Launch Service of LLM-as-a-Judge
Prepare Dataset
Evaluation
CRUD (Chinese dataset)
Prepare Dataset
Launch Service of RAG System
Evaluation
Acknowledgements
🚀 RAG Pilot - A RAG Pipeline Tuning Tool
📖 Overview
🌐 Quickstart Guide
⚙️ Dependencies and Environment Setup
🚀 RAG Pilot - A RAG Pipeline Tuning Tool
📖 Overview
🧠 Available Tuners
🚦 How to use RAG Pilot
🔹Input file: QA List File (
your_queries.csv
)
▶️ Use RAG Pilot with UI
▶️ Use RAG Pilot with RESTful API
🔧 How to Adjust RAG Pilot to Tune Your RAG Solution
🧩 What’s Nodes and Modules
⚙️ How to Configure Nodes and Modules
🧑💻 How to Use Nodes and Modules
Toxicity Detection Accuracy
Get Started on Gaudi 2 Accelerator
Requirements
Setup
Evaluation
Get Started on CPU
Requirements
Installation
Evaluation
Metric Card for BLEU
Metric Description
Intended Uses
How to Use
Inputs
Output Values
Examples
Limitations and Bias
Citation
Further References
RAGAAF (RAG assessment - Annotation Free)
Key features
Run RAGAAF
1. Data
2. Launch endpoint on Gaudi
3. Model
4. Metrics
5. Evaluation
Customizations
OPEA adaption of ragas (LLM-as-a-judge evaluation of Retrieval Augmented Generation)
User data
Launch HuggingFace endpoint on Intel’s Gaudi machines
Run OPEA ragas pipeline using your desired list of metrics
Troubleshooting
Developer Guides
Coding Guides
OPEA API Service Spec (v1.0)
OPEA Mega Service API
OPEA Micro Service API
Documentation Guides
Documentation Guidelines
Markdown vs. RestructuredText
Documentation Organization
Headings
Content Highlighting
Lists
Multi-Column Lists
Tables
File Names and Commands
Branch-Specific File Links
Internal Cross-Reference Linking
Non-ASCII Characters
Include Content from Other Files
Code and Command Examples
Images
Tabs, Spaces, and Indenting
Background Colors
Drawings
Alternative Tabbed Content
Instruction Steps
First Instruction Step
Second Instruction Step
Documentation Generation
README Guidelines for GenAIExamples
Prepare the Primary README File
Prepare the Secondary README File
Drawings Using Graphviz
Simple Directed Graph
Adding Edge Labels
Tables
Finite-State Machine
OPEA Documentation Generation
Documentation Overview
Set Up the Documentation Working Folders
Install the Documentation Tools
Documentation Presentation Theme
Run the Documentation Processors
Doc Build Troubleshooting
Publish Content
Document Versioning
Filter Expected Warnings
OPEA Community
Community Support
Resources
Contributing Guides
Contribution Guidelines
All The Ways To Contribute
Support
Contributor Covenant Code of Conduct
Contributing A New Vector Database to OPEA
Contributing a new Vector Database to GenAIComps
ChatQnA GenAIExample Vector Database Usage
OPEA Project Code Owners
docs Repository Code Owners
GenAIComps Repository Code Owners
GenAIEval Repository Code Owners
GenAIExamples Repository Code Owners
GenAIInfra Repository Code Owners
Continuous Integration (CICD) owners
OPEA Emeritus Code Owners
Reporting a Vulnerability
Script Usage Notice
Roadmaps
OPEA 2024 - 2025 Roadmap
May 2024
June 2024
July 2024
Aug 2024
Sep 2024
Q4 2024
OPEA 2025 Roadmap
Release Cadence
Q1 2025 (v1.2 release)
Q2 2025 (v1.3 release)
Q3 2025 (v1.4 release)
Q4 2025 (v1.5 release)
Q1 2026 (v1.6 release)
OPEA CI/CD Roadmap
Milestone 1 (May, Done)
Milestone 2 (June)
Milestone 3 (July)
Milestone 4 (Aug)
Project Governance
Technical Charter (the “Charter”) for OPEA a Series of LF Projects, LLC
1. Mission and Scope of the Project
2. Technical Steering Committee
3. TSC Voting
4. Compliance with Policies
5. Community Assets
6. General Rules and Operations.
7. Intellectual Property Policy
8. Amendments
Technical Steering Committee (TSC)
Technical Steering Committee Members
Contributor Covenant Code of Conduct
Our Pledge
Our Standards
Enforcement Responsibilities
Scope
Enforcement
Enforcement Guidelines
Attribution
Reporting a Vulnerability
Script Usage Notice
RFC Proposals
Request for Comments (RFCs)
24-05-16 GenAIExamples-001 Using MicroService to Implement ChatQnA
24-05-16 OPEA-001 Overall Design
24-05-24 OPEA-001 Code Structure
24-06-21-OPEA-001-DocSum_Video_Audio
24-06-21-OPEA-001-Guardrails-Gateway
24-07-11-OPEA-Agent
24-08-02-OPEA-AIAvatarChatbot
24-08-05 OPEA Workflow Executor Example
24-08-07 OPEA-001 OPEA GenAIStudio
24-08-20-OPEA-001-AI Gateway API
24-08-21-GenAIExample-002-Edge Craft RAG
24-10-02-GenAIExamples-001-Image_and_Audio_Support_in_MultimodalQnA
24-10-20-OPEA-001-Haystack-Integration
24-11-25-GenAIExamples-Ollama_Support_for_CPU_Server
Purpose
Design
Routing AgentRFC
RFC: OPEA Inference Microservices (OIM)
25-03-14-GenAIExamples-001-CodeTrans-with-Agents
Add Streaming Automatic Speech Recognition (ASR) support in GenAIComps
25-15-01-GenAIExamples-001-Code-Generation-Using-RAG-and-Agents
HybridRAG
Support Air-gapped environment
RFC Template
Publications
News
Events
Blogs
Demos and Videos
Release Notes
OPEA Release Guide
Release Cadence
General Overview
Feature Freeze
Code/Doc Freeze, and Creating the RC Branch
Preparing Creating RC Branch
Creating RC Branch
Cherry Pick Critical Code/Doc Fix
How to do Cherry Picking
Creating Tag from RC Branch
Deliver Docker Images, Helm Charts, and PyPi Binaries
OPEA Release Notes v1.3
Table of Contents
What’s New in OPEA v1.3
Advanced Agent Capabilities
Performance and Scalability
Ecosystem Integrations
New GenAI Capabilities
Enhanced Evaluation
Observability
Better User Experience
Exploration
Newly Supported Models
Newly Supported Hardware
Other Notable Changes
Deprecations
Deprecated Examples
Deprecated Docker Images
Deprecated GenAIExample Variables
Deprecated GenAIComps Parameters
Updated Dependencies
Changes to Default Behavior
Validated Hardware
Validated Software
Known Issues
Full Changelogs
Contributors
Contributing Organizations
Individual Contributors
OPEA Release Notes v1.2
What’s New in OPEA v1.2
Highlights
Deprecations and Behavior Changes
GenAIComps
GenAIExamples
GenAIEval
GenAIInfra
Docker Images
Notable Changes
Full Changelogs
Contributors
Contributing Organizations
Individual Contributors
OPEA Release Notes v1.1
What’s New in OPEA v1.1
Highlights
Notable Changes
Full Changelogs
Contributors
Contributing Organizations
Individual Contributors
OPEA Release Notes v1.0
What’s New in OPEA v1.0
Details
OPEA Release Notes v0.9
What’s New in OPEA v0.9
Details
OPEA Release Notes v0.8
What’s New in OPEA v0.8
Details
Thanks to these contributors
OPEA Release Notes v0.7
OPEA Highlights
GenAIExamples
GenAIComps
GenAIEvals
GenAIInfra
OPEA Release Notes v0.6
OPEA Highlight
GenAIExamples
GenAIComps
GenAIEvals
GenAIInfra
Contributing to OPEA
Additional Content
OPEA Frequently Asked Questions
What is OPEA’s mission?
What is OPEA?
What problems are faced by GenAI deployments within the enterprise?
Why now?
How does it compare to other options for deploying Gen AI solutions within the enterprise?
Will OPEA reference implementations work with proprietary components?
What does OPEA acronym stand for?
How do I pronounce OPEA?
What initial companies and open-source projects joined OPEA?
What is Intel contributing?
When you say Technical Conceptual Framework, what components are included?
What are the different ways partners can contribute to OPEA?
Where can partners see the latest draft of the Conceptual Framework spec?
Is there a cost for joining?
Do I need to be a Linux Foundation member to join?
Where can I report a bug or vulnerability?
OPEA™
Latest
»
GenAI Examples
»
Examples
»
EdgeCraftRAG one-click deployment script
View page source
EdgeCraftRAG one-click deployment script
¶
run quick_start.sh
¶
bash
quick_start.sh