OPEA™ Logo
1.0
OPEA Project v: 1.0
Document Versions
latest
OPEA Project links
Project Home
Wiki
  • Documentation Home
  • OPEA Overview
    • OPEA Project Architecture
      • Microservices: Flexible and Scalable Architecture
      • Megaservices: A Comprehensive Solution
      • Gateways: Customized Access to Mega- and Microservices
    • Next Step
      • Open Platform for Enterprise AI (OPEA) Framework Draft Proposal
  • Getting Started
  • GenAI Examples
    • ChatQnA Sample Guide
      • Overview
      • Purpose
      • Key Implementation Details
      • How It Works
      • Architecture
      • Deployment
      • Troubleshooting
      • Monitoring
    • Generative AI Examples
      • Introduction
      • Architecture
      • Getting Started
      • Supported Examples
      • Contributing to OPEA
      • Additional Content
    • Legal Information
      • License
      • Citation
    • Docker Images
      • Example images
      • Microservice images
    • Supported Examples
      • ChatQnA
    • AgentQnA Application
      • Agents for Question Answering
      • Retrieval tool for agent
    • AudioQnA Application
      • AudioQnA Application
      • AudioQnA accuracy Evaluation
      • Build Mega Service of AudioQnA on Xeon
      • Build Mega Service of AudioQnA on Gaudi
      • Deploy AudioQnA in a Kubernetes Cluster
      • Deploy AudioQnA in Kubernetes Cluster on Xeon and Gaudi
      • AudioQnA
    • ChatQnA Application
      • ChatQnA Application
      • ChatQnA Benchmarking
      • Build Mega Service of ChatQnA on AIPC
      • Build Mega Service of ChatQnA on Xeon
      • Build Mega Service of ChatQnA (with Qdrant) on Xeon
      • Build MegaService of ChatQnA on Gaudi
      • How to Check and Validate Micro Service in the GenAI Example
      • Build MegaService of ChatQnA on NVIDIA GPU
      • Deploy ChatQnA in Kubernetes Cluster
      • Deploy ChatQnA in Kubernetes Cluster on Xeon and Gaudi
      • ChatQnA Conversational UI
      • ChatQnA Customized UI
    • CodeGen Application
      • Code Generation Application
      • CodeGen accuracy Evaluation
      • Build MegaService of CodeGen on Xeon
      • Build MegaService of CodeGen on Gaudi
      • Deploy CodeGen in Kubernetes Cluster
      • Deploy CodeGen in a Kubernetes Cluster
      • Deploy CodeGen with ReactUI
      • Code Gen
      • Code Gen
    • CodeTrans Application
      • Code Translation Application
      • Build Mega Service of CodeTrans on Xeon
      • Build Mega Service of CodeTrans on Gaudi
      • Deploy CodeTrans in Kubernetes Cluster
      • Deploy CodeTrans in a Kubernetes Cluster
      • Code Translation
    • DocIndexRetriever Application
      • DocRetriever Application
      • DocRetriever Application with Docker
      • DocRetriever Application with Docker
    • DocSum Application
      • Document Summarization Application
      • Build Mega Service of Document Summarization on Intel Xeon Processor
      • Build MegaService of Document Summarization on Gaudi
      • Deploy DocSum in Kubernetes Cluster
      • Deploy DocSum in Kubernetes Cluster
      • Deploy DocSum with ReactUI
      • Doc Summary React
      • Doc Summary
    • FaqGen Application
      • FAQ Generation Application
      • FaqGen Evaluation
      • Build Mega Service of FAQ Generation on Intel Xeon Processor
      • Build MegaService of FAQ Generation on Gaudi
      • Deploy FaqGen in Kubernetes Cluster
      • Deploy FaqGen with ReactUI
      • Doc Summary React
      • FAQ Generation
    • InstructionTuning Application
      • Instruction Tuning
      • Deploy Instruction Tuning Service on Xeon
      • Deploy Instruction Tuning Service on Gaudi
    • MultimodalQnA Application
      • MultimodalQnA Application
      • Build Mega Service of MultimodalQnA on Xeon
      • Build Mega Service of MultimodalQnA on Gaudi
    • ProductivitySuite Application
      • OPEA Productivity Suite Application
      • Build Mega Service of Productivity Suite on Xeon
      • Keycloak Configuration Setup
      • Deploy ProductivitySuite with ReactUI
      • Productivity Suite React UI
    • RerankFinetuning Application
      • Rerank Model Finetuning
      • Deploy Rerank Model Finetuning Service on Xeon
      • Deploy Rerank Model Finetuning Service on Gaudi
    • SearchQnA Application
      • SearchQnA Application
      • Build Mega Service of SearchQnA on Xeon
      • Build Mega Service of SearchQnA on Gaudi
      • Deploy SearchQnA in a Kubernetes Cluster
      • Neural Chat
    • Translation Application
      • Translation Application
      • Build Mega Service of Translation on Xeon
      • Build MegaService of Translation on Gaudi
      • Deploy Translation in Kubernetes Cluster
      • Deploy Translation in a Kubernetes Cluster
      • Language Translation
    • VideoQnA Application
      • VideoQnA Application
      • Build Mega Service of VideoQnA on Xeon
    • VisualQnA Application
      • Visual Question and Answering
      • Build Mega Service of VisualQnA on Xeon
      • Build MegaService of VisualQnA on Gaudi
      • Deploy VisualQnA in Kubernetes Cluster
      • Deploy VisualQnA in a Kubernetes Cluster
  • GenAI Microservices
    • Generative AI Components (GenAIComps)
      • GenAIComps
      • MicroService
      • MegaService
      • Gateway
      • Contributing to OPEA
      • Additional Content
    • Legal Information
      • License
      • Citation
    • Agent Microservice
      • Agent Microservice
      • Plan Execute
      • RAG Agent
    • Asr Microservice
      • ASR Microservice
    • Chathistory Microservice
      • Chat History Microservice
    • Cores Microservice
      • Telemetry for OPEA
    • Dataprep Microservice
      • Dataprep Microservice
      • Dataprep Microservice with Milvus
      • Dataprep Microservice for Multimodal Data with Redis
      • Dataprep Microservice with Neo4J
      • Dataprep Microservice with PGVector
      • Dataprep Microservice with Pinecone
      • Dataprep Microservice with Qdrant
      • Dataprep Microservice with Redis
      • Dataprep Microservice with VDMS
      • Multimodal Dataprep Microservice with VDMS
    • Embeddings Microservice
      • Embeddings Microservice
      • build Mosec endpoint docker image
      • Embedding Server
      • Multimodal Embeddings Microservice
      • Multimodal CLIP Embeddings Microservice
      • Embedding Generation Prediction Guard Microservice
      • Embeddings Microservice with Langchain TEI
      • Embeddings Microservice with Llama Index TEI
    • Feedback_management Microservice
      • Feedback Management Microservice
    • Finetuning Microservice
      • Fine-tuning Microservice
    • Guardrails Microservice
      • Trust and Safety with LLM
      • Bias Detection Microservice
      • Guardrails Microservice
      • PII Detection Microservice
      • Toxicity Detection Microservice
    • Intent_detection Microservice
      • Intent Detection Microservice by TGI
    • Knowledgegraphs Microservice
      • Knowledge Graph Microservice
    • Llms Microservice
      • TGI FAQGen LLM Microservice
      • Document Summary TGI Microservice
      • LLM Microservice
      • LLM Native Microservice
      • LLM Native Microservice
      • Introduction
      • Prediction Guard Introduction
      • TGI LLM Microservice
      • vLLM Endpoint Service
      • vLLM Endpoint Service
      • VLLM-Ray Endpoint Service
      • LM-Eval Microservice
    • Lvms Microservice
      • LVM Microservice
      • LVM Prediction Guard Microservice
      • LVM Microservice
    • Nginx Microservice
      • Nginx for Microservice Forwarding
    • Prompt_registry Microservice
      • Prompt Registry Microservice
    • Ragas Microservice
    • Reranks Microservice
      • Reranking Microservice
      • build reranking Mosec endpoint docker image
      • Reranking Microservice
      • Rerank Microservice
    • Retrievers Microservice
      • Retriever Microservice
      • Retriever Microservice with Milvus
      • Retriever Microservice
      • Retriever Microservice with Neo4J
      • Retriever Microservice with Pathway
      • Retriever Microservice
      • Retriever Microservice with Qdrant
      • Retriever Microservice
      • Retriever Microservice
      • Retriever Microservice
    • Tts Microservice
      • TTS Microservice
    • Vectorstores Microservice
      • Vectorstores Microservice
      • Start Chroma server
      • Start LanceDB Server
      • Start Milvus server
      • Start the Pathway Vector DB Server
      • Start PGVector server
      • Pinecone setup
      • Start Qdrant server
      • Start Redis server
      • Start VDMS server
    • Web_retrievers Microservice
      • Web Retriever Microservice
  • Deploying GenAI
    • GenAIInfra
      • Overview
      • Prerequisite
      • Usages
      • Additional Content
    • Development
      • Prerequisites
      • Testing
      • pre-commit testing
    • Legal Information
      • License
      • Citation
    • Release Branches
      • 1. Create release candidate branch
      • 2. Create images with release tag
      • 3. Test helm charts
      • 4. Test GMC
      • 5. Publish images
    • Installation Guides
      • GenAI-microservices-connector(GMC) Installation
      • Kubernetes Installation using AWS EKS Cluster
      • Kubernetes installation demo using kubeadm
      • Kubernetes installation using Kubespray
    • Authentication and Authorization
      • Authentication and authorization
      • Authentication and Authorization with APISIX and OIDC based Identity provider (Keycloak)
      • Leveraging Istio to compose an OPEA Pipeline with authentication and authorization enabled
    • Helm Charts
      • Helm charts for deploying GenAI Components and Examples
      • HorizontalPodAutoscaler (HPA) support
      • ChatQnA
      • ChatQnA Troubleshooting
      • CodeGen
      • CodeTrans
      • asr
      • data-prep
      • embedding-usvc
      • guardrails-usvc
      • llm-uservice
      • redis-vector-db
      • reranking-usvc
      • retriever-usvc
      • speecht5
      • tei
      • teirerank
      • tgi
      • tts
      • vllm
      • web-retriever
      • whisper
      • DocSum
    • Kubernetes Addons
      • Deploy Kubernetes add-ons for OPEA
      • Intel® Gaudi® Base Operator for Kubernetes
      • How-To Setup Observability for OPEA Workload in Kubernetes
      • memory bandwidth exporter
    • Microservices Connector
      • genai-microservices-connector(GMC)
      • Troubleshooting GMC Custom Resource(CR)
      • Usage guide for genai-microservices-connector(GMC)
      • ChatQnA Use Cases in Kubernetes Cluster via GMC
      • Helm chart for genai-microservices-connector(GMC)
    • Pipeline Proxy
      • OPEA Pipeline Proxy
      • Guardrails
    • Scripts
      • Scripts and tools
      • NVIDIA GPU Quick-Start Guide
      • Deploy Autoscaling Ray Cluster with KubeRay in Kubernetes Cluster
  • Evaluating GenAI
    • GenAIEval
      • Installation
      • Evaluation
      • Benchmark
      • Additional Content
    • Legal Information
      • License
      • Citation
    • Kubernetes Platform Optimization with Resource Management
      • Introduction
      • NRI Plugins
      • Install
      • Validate policy status
      • Configure
      • Validate CPU affinity and hardware alignment in containers
      • Remove a policy
      • NRI topology-aware resource policy
    • OPEA Benchmark Tool
      • Features
      • Table of Contents
      • Installation
      • Usage
      • Configuration
    • Auto-Tuning for ChatQnA: Optimizing Resource Allocation in Kubernetes
      • Key Features
    • Usage
      • Configuration Files
      • Output
    • Auto-Tuning for ChatQnA: Optimizing Accuracy by Tuning Model Related Parameters
      • Prepare Dataset
      • Run the Tuning script
    • Setup Prometheus and Grafana to visualize microservice metrics
      • 1. Setup Prometheus
      • 2. Setup Grafana
    • StressCli
      • stresscli.py
    • locust scripts for OPEA ChatQnA
      • Configuration file
      • Basic Usage
    • CRAG Benchmark for Agent QnA systems
      • Overview
      • Getting started
      • CRAG dataset
      • Launch agent QnA system
      • Run CRAG benchmark
      • Use LLM-as-judge to grade the answers
    • AutoRAG to evaluate the RAG system performance
      • Service preparation
      • RAG evaluation
      • Notes
    • Evaluation Methodology
      • Introduction
      • Prerequisite
      • MultiHop (English dataset)
      • CRUD (Chinese dataset)
      • Acknowledgements
    • Metric Card for BLEU
      • Metric Description
      • Intended Uses
      • How to Use
      • Limitations and Bias
      • Citation
      • Further References
  • Developer Guides
    • Coding Guides
      • OPEA API Service Spec (v0.9)
    • Documentation Guides
      • Documentation Guidelines
      • Drawings Using Graphviz
      • OPEA Documentation Generation
  • OPEA Community
    • Community Support
    • Resources
    • Contributing Guides
      • OPEA Project
      • Contribution Guidelines
      • OPEA Project Code Owners
      • Reporting a Vulnerability
      • Documentation Guidelines
      • OPEA Documentation Generation
      • Drawings Using Graphviz
    • Roadmaps
      • OPEA 2024 - 2025 Roadmap
      • OPEA CI/CD Roadmap
    • Project Governance
      • Technical Charter (the “Charter”) for OPEA a Series of LF Projects, LLC
      • Technical Steering Committee (TSC)
      • Contributor Covenant Code of Conduct
      • Reporting a Vulnerability
    • RFC Proposals
      • Request for Comments (RFCs)
  • Release Notes
    • OPEA Release Notes v1.0
      • What’s New in OPEA v1.0
      • Details
    • OPEA Release Notes v0.9
      • What’s New in OPEA v0.9
      • Details
    • OPEA Release Notes v0.8
      • What’s New in OPEA v0.8
      • Details
      • Thanks to these contributors
    • OPEA Release Notes v0.7
      • OPEA Highlights
      • GenAIExamples
      • GenAIComps
      • GenAIEvals
      • GenAIInfra
    • OPEA Release Notes v0.6
      • OPEA Highlight
      • GenAIExamples
      • GenAIComps
      • GenAIEvals
      • GenAIInfra
  • OPEA Frequently Asked Questions
    • What is OPEA’s mission?
    • What is OPEA?
    • What problems are faced by GenAI deployments within the enterprise?
    • Why now?
    • How does it compare to other options for deploying Gen AI solutions within the enterprise?
    • Will OPEA reference implementations work with proprietary components?
    • What does OPEA acronym stand for?
    • How do I pronounce OPEA?
    • What initial companies and open-source projects joined OPEA?
    • What is Intel contributing?
    • When you say Technical Conceptual Framework, what components are included?
    • What are the different ways partners can contribute to OPEA?
    • Where can partners see the latest draft of the Conceptual Framework spec?
    • Is there a cost for joining?
    • Do I need to be a Linux Foundation member to join?
    • Where can I report a bug or vulnerability?
OPEA™
  • 1.0 »
  • Deploying GenAI »
  • Legal Information
  • View page source

Legal Information¶

  1. License

  2. Citation

License¶

Generative AI Examples is licensed under Apache License Version 2.0. This software includes components that have separate copyright notices and licensing terms. Your use of the source code for these components is subject to the terms and conditions of the following licenses.

See the accompanying license file for full license text and copyright notices.

Citation¶

If you use Generative AI Infrastructure in your research, use the following BibTeX entry.

@misc{Generative AI Infrastructure,
  author =       {Jianfeng Ding},
  title =        {Generative AI Infrastructure},
  howpublished = {\url{https://github.com/opea-project/GenAIInfra}},
  year =         {2024}
}
Previous Next

© Copyright 2024 OPEA™, a Series of LF Projects, LLC. Published on Sep 20, 2024.