OPEA™ Logo
1.0
OPEA Project v: 1.0
Document Versions
latest
OPEA Project links
Project Home
Wiki
  • Documentation Home
  • OPEA Overview
    • OPEA Project Architecture
      • Microservices: Flexible and Scalable Architecture
      • Megaservices: A Comprehensive Solution
      • Gateways: Customized Access to Mega- and Microservices
    • Next Step
      • Open Platform for Enterprise AI (OPEA) Framework Draft Proposal
  • Getting Started
  • GenAI Examples
    • ChatQnA Sample Guide
      • Overview
      • Purpose
      • Key Implementation Details
      • How It Works
      • Architecture
      • Deployment
      • Troubleshooting
      • Monitoring
    • Generative AI Examples
      • Introduction
      • Architecture
      • Getting Started
      • Supported Examples
      • Contributing to OPEA
      • Additional Content
    • Legal Information
      • License
      • Citation
    • Docker Images
      • Example images
      • Microservice images
    • Supported Examples
      • ChatQnA
    • AgentQnA Application
      • Agents for Question Answering
      • Retrieval tool for agent
    • AudioQnA Application
      • AudioQnA Application
      • AudioQnA accuracy Evaluation
      • Build Mega Service of AudioQnA on Xeon
      • Build Mega Service of AudioQnA on Gaudi
      • Deploy AudioQnA in a Kubernetes Cluster
      • Deploy AudioQnA in Kubernetes Cluster on Xeon and Gaudi
      • AudioQnA
    • ChatQnA Application
      • ChatQnA Application
      • ChatQnA Benchmarking
      • Build Mega Service of ChatQnA on AIPC
      • Build Mega Service of ChatQnA on Xeon
      • Build Mega Service of ChatQnA (with Qdrant) on Xeon
      • Build MegaService of ChatQnA on Gaudi
      • How to Check and Validate Micro Service in the GenAI Example
      • Build MegaService of ChatQnA on NVIDIA GPU
      • Deploy ChatQnA in Kubernetes Cluster
      • Deploy ChatQnA in Kubernetes Cluster on Xeon and Gaudi
      • ChatQnA Conversational UI
      • ChatQnA Customized UI
    • CodeGen Application
      • Code Generation Application
      • CodeGen accuracy Evaluation
      • Build MegaService of CodeGen on Xeon
      • Build MegaService of CodeGen on Gaudi
      • Deploy CodeGen in Kubernetes Cluster
      • Deploy CodeGen in a Kubernetes Cluster
      • Deploy CodeGen with ReactUI
      • Code Gen
      • Code Gen
    • CodeTrans Application
      • Code Translation Application
      • Build Mega Service of CodeTrans on Xeon
      • Build Mega Service of CodeTrans on Gaudi
      • Deploy CodeTrans in Kubernetes Cluster
      • Deploy CodeTrans in a Kubernetes Cluster
      • Code Translation
    • DocIndexRetriever Application
      • DocRetriever Application
      • DocRetriever Application with Docker
      • DocRetriever Application with Docker
    • DocSum Application
      • Document Summarization Application
      • Build Mega Service of Document Summarization on Intel Xeon Processor
      • Build MegaService of Document Summarization on Gaudi
      • Deploy DocSum in Kubernetes Cluster
      • Deploy DocSum in Kubernetes Cluster
      • Deploy DocSum with ReactUI
      • Doc Summary React
      • Doc Summary
    • FaqGen Application
      • FAQ Generation Application
      • FaqGen Evaluation
      • Build Mega Service of FAQ Generation on Intel Xeon Processor
      • Build MegaService of FAQ Generation on Gaudi
      • Deploy FaqGen in Kubernetes Cluster
      • Deploy FaqGen with ReactUI
      • Doc Summary React
      • FAQ Generation
    • InstructionTuning Application
      • Instruction Tuning
      • Deploy Instruction Tuning Service on Xeon
      • Deploy Instruction Tuning Service on Gaudi
    • MultimodalQnA Application
      • MultimodalQnA Application
      • Build Mega Service of MultimodalQnA on Xeon
      • Build Mega Service of MultimodalQnA on Gaudi
    • ProductivitySuite Application
      • OPEA Productivity Suite Application
      • Build Mega Service of Productivity Suite on Xeon
      • Keycloak Configuration Setup
      • Deploy ProductivitySuite with ReactUI
      • Productivity Suite React UI
    • RerankFinetuning Application
      • Rerank Model Finetuning
      • Deploy Rerank Model Finetuning Service on Xeon
      • Deploy Rerank Model Finetuning Service on Gaudi
    • SearchQnA Application
      • SearchQnA Application
      • Build Mega Service of SearchQnA on Xeon
      • Build Mega Service of SearchQnA on Gaudi
      • Deploy SearchQnA in a Kubernetes Cluster
      • Neural Chat
    • Translation Application
      • Translation Application
      • Build Mega Service of Translation on Xeon
      • Build MegaService of Translation on Gaudi
      • Deploy Translation in Kubernetes Cluster
      • Deploy Translation in a Kubernetes Cluster
      • Language Translation
    • VideoQnA Application
      • VideoQnA Application
      • Build Mega Service of VideoQnA on Xeon
    • VisualQnA Application
      • Visual Question and Answering
      • Build Mega Service of VisualQnA on Xeon
      • Build MegaService of VisualQnA on Gaudi
      • Deploy VisualQnA in Kubernetes Cluster
      • Deploy VisualQnA in a Kubernetes Cluster
  • GenAI Microservices
    • Generative AI Components (GenAIComps)
      • GenAIComps
      • MicroService
      • MegaService
      • Gateway
      • Contributing to OPEA
      • Additional Content
    • Legal Information
      • License
      • Citation
    • Agent Microservice
      • Agent Microservice
      • Plan Execute
      • RAG Agent
    • Asr Microservice
      • ASR Microservice
    • Chathistory Microservice
      • Chat History Microservice
    • Cores Microservice
      • Telemetry for OPEA
    • Dataprep Microservice
      • Dataprep Microservice
      • Dataprep Microservice with Milvus
      • Dataprep Microservice for Multimodal Data with Redis
      • Dataprep Microservice with Neo4J
      • Dataprep Microservice with PGVector
      • Dataprep Microservice with Pinecone
      • Dataprep Microservice with Qdrant
      • Dataprep Microservice with Redis
      • Dataprep Microservice with VDMS
      • Multimodal Dataprep Microservice with VDMS
    • Embeddings Microservice
      • Embeddings Microservice
      • build Mosec endpoint docker image
      • Embedding Server
      • Multimodal Embeddings Microservice
      • Multimodal CLIP Embeddings Microservice
      • Embedding Generation Prediction Guard Microservice
      • Embeddings Microservice with Langchain TEI
      • Embeddings Microservice with Llama Index TEI
    • Feedback_management Microservice
      • Feedback Management Microservice
    • Finetuning Microservice
      • Fine-tuning Microservice
    • Guardrails Microservice
      • Trust and Safety with LLM
      • Bias Detection Microservice
      • Guardrails Microservice
      • PII Detection Microservice
      • Toxicity Detection Microservice
    • Intent_detection Microservice
      • Intent Detection Microservice by TGI
    • Knowledgegraphs Microservice
      • Knowledge Graph Microservice
    • Llms Microservice
      • TGI FAQGen LLM Microservice
      • Document Summary TGI Microservice
      • LLM Microservice
      • LLM Native Microservice
      • LLM Native Microservice
      • Introduction
      • Prediction Guard Introduction
      • TGI LLM Microservice
      • vLLM Endpoint Service
      • vLLM Endpoint Service
      • VLLM-Ray Endpoint Service
      • LM-Eval Microservice
    • Lvms Microservice
      • LVM Microservice
      • LVM Prediction Guard Microservice
      • LVM Microservice
    • Nginx Microservice
      • Nginx for Microservice Forwarding
    • Prompt_registry Microservice
      • Prompt Registry Microservice
    • Ragas Microservice
    • Reranks Microservice
      • Reranking Microservice
      • build reranking Mosec endpoint docker image
      • Reranking Microservice
      • Rerank Microservice
    • Retrievers Microservice
      • Retriever Microservice
      • Retriever Microservice with Milvus
      • Retriever Microservice
      • Retriever Microservice with Neo4J
      • Retriever Microservice with Pathway
      • Retriever Microservice
      • Retriever Microservice with Qdrant
      • Retriever Microservice
      • Retriever Microservice
      • Retriever Microservice
    • Tts Microservice
      • TTS Microservice
    • Vectorstores Microservice
      • Vectorstores Microservice
      • Start Chroma server
      • Start LanceDB Server
      • Start Milvus server
      • Start the Pathway Vector DB Server
      • Start PGVector server
      • Pinecone setup
      • Start Qdrant server
      • Start Redis server
      • Start VDMS server
    • Web_retrievers Microservice
      • Web Retriever Microservice
  • Deploying GenAI
    • GenAIInfra
      • Overview
      • Prerequisite
      • Usages
      • Additional Content
    • Development
      • Prerequisites
      • Testing
      • pre-commit testing
    • Legal Information
      • License
      • Citation
    • Release Branches
      • 1. Create release candidate branch
      • 2. Create images with release tag
      • 3. Test helm charts
      • 4. Test GMC
      • 5. Publish images
    • Installation Guides
      • GenAI-microservices-connector(GMC) Installation
      • Kubernetes Installation using AWS EKS Cluster
      • Kubernetes installation demo using kubeadm
      • Kubernetes installation using Kubespray
    • Authentication and Authorization
      • Authentication and authorization
      • Authentication and Authorization with APISIX and OIDC based Identity provider (Keycloak)
      • Leveraging Istio to compose an OPEA Pipeline with authentication and authorization enabled
    • Helm Charts
      • Helm charts for deploying GenAI Components and Examples
      • HorizontalPodAutoscaler (HPA) support
      • ChatQnA
      • ChatQnA Troubleshooting
      • CodeGen
      • CodeTrans
      • asr
      • data-prep
      • embedding-usvc
      • guardrails-usvc
      • llm-uservice
      • redis-vector-db
      • reranking-usvc
      • retriever-usvc
      • speecht5
      • tei
      • teirerank
      • tgi
      • tts
      • vllm
      • web-retriever
      • whisper
      • DocSum
    • Kubernetes Addons
      • Deploy Kubernetes add-ons for OPEA
      • Intel® Gaudi® Base Operator for Kubernetes
      • How-To Setup Observability for OPEA Workload in Kubernetes
      • memory bandwidth exporter
    • Microservices Connector
      • genai-microservices-connector(GMC)
      • Troubleshooting GMC Custom Resource（CR）
      • Usage guide for genai-microservices-connector(GMC)
      • ChatQnA Use Cases in Kubernetes Cluster via GMC
      • Helm chart for genai-microservices-connector(GMC)
    • Pipeline Proxy
      • OPEA Pipeline Proxy
      • Guardrails
    • Scripts
      • Scripts and tools
      • NVIDIA GPU Quick-Start Guide
      • Deploy Autoscaling Ray Cluster with KubeRay in Kubernetes Cluster
  • Evaluating GenAI
    • GenAIEval
      • Installation
      • Evaluation
      • Benchmark
      • Additional Content
    • Legal Information
      • License
      • Citation
    • Kubernetes Platform Optimization with Resource Management
      • Introduction
      • NRI Plugins
      • Install
      • Validate policy status
      • Configure
      • Validate CPU affinity and hardware alignment in containers
      • Remove a policy
      • NRI topology-aware resource policy
    • OPEA Benchmark Tool
      • Features
      • Table of Contents
      • Installation
      • Usage
      • Configuration
    • Auto-Tuning for ChatQnA: Optimizing Resource Allocation in Kubernetes
      • Key Features
    • Usage
      • Configuration Files
      • Output
    • Auto-Tuning for ChatQnA: Optimizing Accuracy by Tuning Model Related Parameters
      • Prepare Dataset
      • Run the Tuning script
    • Setup Prometheus and Grafana to visualize microservice metrics
      • 1. Setup Prometheus
      • 2. Setup Grafana
    • StressCli
      • stresscli.py
    • locust scripts for OPEA ChatQnA
      • Configuration file
      • Basic Usage
    • CRAG Benchmark for Agent QnA systems
      • Overview
      • Getting started
      • CRAG dataset
      • Launch agent QnA system
      • Run CRAG benchmark
      • Use LLM-as-judge to grade the answers
    • AutoRAG to evaluate the RAG system performance
      • Service preparation
      • RAG evaluation
      • Notes
    • Evaluation Methodology
      • Introduction
      • Prerequisite
      • MultiHop (English dataset)
      • CRUD (Chinese dataset)
      • Acknowledgements
    • Metric Card for BLEU
      • Metric Description
      • Intended Uses
      • How to Use
      • Limitations and Bias
      • Citation
      • Further References
  • Developer Guides
    • Coding Guides
      • OPEA API Service Spec (v0.9)
    • Documentation Guides
      • Documentation Guidelines
      • Drawings Using Graphviz
      • OPEA Documentation Generation
  • OPEA Community
    • Community Support
    • Resources
    • Contributing Guides
      • OPEA Project
      • Contribution Guidelines
      • OPEA Project Code Owners
      • Reporting a Vulnerability
      • Documentation Guidelines
      • OPEA Documentation Generation
      • Drawings Using Graphviz
    • Roadmaps
      • OPEA 2024 - 2025 Roadmap
      • OPEA CI/CD Roadmap
    • Project Governance
      • Technical Charter (the “Charter”) for OPEA a Series of LF Projects, LLC
      • Technical Steering Committee (TSC)
      • Contributor Covenant Code of Conduct
      • Reporting a Vulnerability
    • RFC Proposals
      • Request for Comments (RFCs)
  • Release Notes
    • OPEA Release Notes v1.0
      • What’s New in OPEA v1.0
      • Details
    • OPEA Release Notes v0.9
      • What’s New in OPEA v0.9
      • Details
    • OPEA Release Notes v0.8
      • What’s New in OPEA v0.8
      • Details
      • Thanks to these contributors
    • OPEA Release Notes v0.7
      • OPEA Highlights
      • GenAIExamples
      • GenAIComps
      • GenAIEvals
      • GenAIInfra
    • OPEA Release Notes v0.6
      • OPEA Highlight
      • GenAIExamples
      • GenAIComps
      • GenAIEvals
      • GenAIInfra
  • OPEA Frequently Asked Questions
    • What is OPEA’s mission?
    • What is OPEA?
    • What problems are faced by GenAI deployments within the enterprise?
    • Why now?
    • How does it compare to other options for deploying Gen AI solutions within the enterprise?
    • Will OPEA reference implementations work with proprietary components?
    • What does OPEA acronym stand for?
    • How do I pronounce OPEA?
    • What initial companies and open-source projects joined OPEA?
    • What is Intel contributing?
    • When you say Technical Conceptual Framework, what components are included?
    • What are the different ways partners can contribute to OPEA?
    • Where can partners see the latest draft of the Conceptual Framework spec?
    • Is there a cost for joining?
    • Do I need to be a Linux Foundation member to join?
    • Where can I report a bug or vulnerability?
OPEA™
  • 1.0 »
  • Search

© Copyright 2024 OPEA™, a Series of LF Projects, LLC. Published on Sep 20, 2024.