Deploying GenAI

GenAIInfra is the containerization and cloud native suite for OPEA, including artifacts to deploy GenAI Examples in a cloud native way so enterprise users can deploy to their own cloud.

We’re building this documentation from content in the GenAIInfra GitHub repository.

• GenAIInfra
• Development
• Legal Information
• Release Branches

Installation Guides

• GenAI-microservices-connector(GMC) Installation
• Kubernetes Installation using AWS EKS Cluster
• Kubernetes installation demo using kubeadm
• Kubernetes installation using Kubespray

Authentication and Authorization

• Authentication and authorization
• Authentication and Authorization with APISIX and OIDC based Identity provider (Keycloak)
• Leveraging Istio to compose an OPEA Pipeline with authentication and authorization enabled

Helm Charts

• Helm charts for deploying GenAI Components and Examples
• HorizontalPodAutoscaler (HPA) support
• ChatQnA
• CodeGen
• CodeTrans
• asr
• data-prep
• embedding-usvc
• guardrails-usvc
• llm-uservice
• redis-vector-db
• reranking-usvc
• retriever-usvc
• speecht5
• tei
• teirerank
• tgi
• tts
• vllm
• web-retriever
• whisper
• DocSum

Kubernetes Addons

• Deploy Kubernetes add-ons for OPEA
• Intel® Gaudi® Base Operator for Kubernetes
• How-To Setup Observability for OPEA Workload in Kubernetes
• memory bandwidth exporter

Microservices Connector

• genai-microservices-connector(GMC)
• Troubleshooting GMC Custom Resource（CR）
• Usage guide for genai-microservices-connector(GMC)
• ChatQnA Use Cases in Kubernetes Cluster via GMC
• Helm chart for genai-microservices-connector(GMC)

Pipeline Proxy

• OPEA Pipeline Proxy
• Guardrails

Scripts

• Scripts and tools
• NVIDIA GPU Quick-Start Guide
• Deploy Autoscaling Ray Cluster with KubeRay in Kubernetes Cluster