StressCli

This project includes benchmark toolset for AI workloads such as OPEA.

stresscli.py

stresscli.py is a command line tool for dumping test specs and performing load tests.

Prerequirements

This tool will use kubectl to collect Kubernetes cluster information. So you need to make sure kubectl is installed and

This tool uses locust by default to do load test. You have to install locust to your machine simply by

pip3 install locust

For detail information of locust, go to locust website.

Installation

The recommended way to install and run stresscli is in a virtualenv with the latest version of Python 3 (at least Python 3.11). If Python is not installed, you can likely install it using your distribution’s package manager, or see the Python Download page.

git clone https://github.com/opea-project/GenAIEval.git
# go to project root
cd GenAIEval/evals/benchmark/stresscli
# create virtual env
python3 -m venv stresscli_virtualenv
source stresscli_virtualenv/bin/activate
# install requirements
pip install -r requirements.txt

Usage

./stresscli.py --help
Usage: stresscli.py [OPTIONS] COMMAND [ARGS]...

  StressCLI - A command line tool for stress testing OPEA workloads.

Options:
  --kubeconfig PATH  Configuration file to Kubernetes
  --help             Show this message and exit.

Commands:
  dump       Dump the test spec
  load-test  Do load test
  report     Print the test report
  validate   Validate against the test spec

Run a test

Note: Please edit the run.yaml file or create your profile before run the load test.

./stresscli.py load-test --profile run.yaml

More detail options:

./stresscli.py load-test --help
Usage: stresscli.py load-test [OPTIONS]

  Do load test

Options:
  --profile PATH  Path to profile YAML file
  --help          Show this message and exit.

Generate the test output report

You can generate the report for test cases by:

./stresscli.py report --folder /home/sdp/test_reports/20240710_004105 --format csv -o data.csv

More detail options:

./stresscli.py report --help
Usage: stresscli.py report [OPTIONS]

  Print the test report

Options:
  --folder PATH      Path to log folder  [required]
  --format TEXT      Output format, plain_text or csv, default is plain_text
  --include TEXT     Extract output data from output.log, stats.csv, and
                     testspec.yaml, default is
                     output.log|stats.csv|testspec.yaml
  -o, --output PATH  Save output to file
  --help             Show this message and exit.

Dump the configuration

You can dump the current testing profile by

./stresscli.py dump -o <output_file>

More detail options:

./stresscli.py dump --help
Usage: stresscli.py dump [OPTIONS]

  Dump the test spec

Options:
  -o, --output PATH  YAML file of cluster configuration  [required]
  --help             Show this message and exit.

Validate against the spec

You can validate if the current K8s and workloads deployment comply with the test spec by:

./stresscli.py validate --file testspec.yaml

More detail options:

./stresscli.py validate --help
Usage: stresscli.py validate [OPTIONS]

  Validate against the test spec

Options:
  --file PATH          Specification YAML file to validate against  [required]
  --validate_topology  Validate topology in workload specification
  --help               Show this message and exit.