# litmus-2.0

**Repository Path**: mirrors_influxdata/litmus-2.0

## Basic Information

- **Project Name**: litmus-2.0
- **Description**: No description available
- **Primary Language**: Unknown
- **License**: Not specified
- **Default Branch**: master
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 0
- **Forks**: 0
- **Created**: 2020-12-16
- **Last Updated**: 2025-08-30

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

Litmus is an automated regression test framework suitable for testing cloud 2.0 offerings (OSS: platform, IDPE: enterprise).  
Currently the framework is not suitable for testing of UI component (chronograf), but it could be extended if needs to be.  
Litmus is based on pytest open source framework and uses Python for writing tests and supporting libraries.

**Litmus** can be run as a `docker container` or from a local repo (`idpe/litmus`)  

## **Building Litmus**
The Litmus framework uses `poetry` to fetch dependencies and manage virtualenvs. See the installation instructions
[here](https://python-poetry.org/docs/#installation). Once installed, run `poetry install` to set up a virtual env
containing the framework's dependencies, and `poetry shell` to drop into the env. All commands listed below assume
you are running inside the virtual env.

### **Building Docker**
Run `make image/build` to create a docker image containing the Litmus tests and all dependencies. The container uses
raw python / pip to manage dependencies. Poetry is still required on your local machine to build the image.

## **Litmus Framework**

- **To display help**
  - By running the command:
  `python scripts/litmus_run_master.py -h`
  
- **Running Litmus**  
  - Litmus accepts either test file(s) or test lists.  
    - Test files (start with **`test_`**) currently organized to group related tests into test classes. One test file might have more than one test class.
  If a feature, for example `tasks`, has multiple sub-features, such as `create task`, `update tasks`, etc; then one test file would be named `test_task_create`,
  another `test_task_update` and etc. Inside the test file (sub-features) tests can be grouped based on sub-sub-features into
  corresponding test classes, e.g. `TestTaskUpdateFlux` class, `TestTaskUpdateAuth`, etc. In order to pass a single test file
  use **`--tests <path to a test file>`**
    - If there is a need to run multiple test files at once, for example, having one test run to test `featureA`, 
  where testing of `featureA` is organized into multiple test files. All of the `tests lists` could be found under 
  `idpe/litmus/tests_lists` directory. In order to pass a list of tests to be executed, 
  use **`--tests-lists <path to a test list file>`**
    - Litmus will create a log file for each test file being executed with following name:
  **`<test file>.log`** in the **`idpe/litmus/result`** directory.
    - **Litmus** can be used to run tests on local development environments (deployed with `kubone`),
  or in our cloud environment (deployed with `fenv`)
    - Whether you are running IDPE or OSS tests, Litmus will do all of the **bootstrapping** for you, i.e. will
  create a `super user/password org/bucket` using `setup` fixture, that returns a `super user's token`.
  This fixture has a `class` scope, that is for each test class it will setUp/tearDown the `super user`
    - Every time when there is a change, Litmus docker image will be created with a `tag:<PR>` or `tag:latest`
  for master builds. Every Litmus image is stored on `quay.io/influxdb/litmus`
    - Every test run (whether open source or idpe) will send a message for every successful or failed test run with a
  link to the build to the `#testing` channel.
  
- **Writing tests**
  - TBD
  
- **How to run Litmus from a docker container**
  - The specific docker image can be used, but mostly you would want to use a `litmus:latest`  
    Litmus docker image is for **`ubuntu`** os and `influxd` binary needs to be build for `ubuntu`  
    **Running on Open Source Platform**  
  - Running One Test using `influxd` binary:
    - >docker run --entrypoint "./run_litmus_tests_oss.sh" -e ONE_TEST=src/cloud/rest_api/<path to test file> -e BINARYPATH=<path_to_/influxd> 
    -e BOLTPATH=<path_to_/influxd.bolt> -e ENGINEPATH=<path_to_engine> --net host -v /var/run/docker.sock:/var/run/docker.sock -v <local_dir>:/Litmus/result quay.io/influxdb/litmus:<tag\>    
  - Running List Of Tests using `influxd` binary:
    - >docker run --entrypoint "./run_litmus_tests_oss.sh" -e TEST_LIST=tests_lists/<test_list_file> -e BINARYPATH=<path_to_/influxd> 
    -e BOLTPATH=<path_to_/influxd.bolt> -e ENGINEPATH=<path to engine> --net host -v /var/run/docker.sock:/var/run/docker.sock -v <local_dir>:/Litmus/result quay.io/influxdb/litmus:<tag\>   
  - Running One Test using `influx` docker image:
    - >docker run --entrypoint "./run_litmus_tests_oss.sh" -e ONE_TEST=src/cloud/rest_api/<path_to_test_file> -e DOCKERIMAGE=true --net host -v /var/run/docker.sock:/var/run/docker.sock 
    -v <local_dir>:/Litmus/result quay.io/influxdb/litmus:<tag\>
  - Running List Of Tests using `influx` docker image:
    - >docker run --entrypoint "./run_litmus_tests_oss.sh" -e TEST_LIST=tests_lists/<test_list_file> -e DOCKERIMAGE=true --net host -v /var/run/docker.sock:/var/run/docker.sock 
    -v <local_dir>:/Litmus/result quay.io/influxdb/litmus:<tag\>  
    
    **Running on IDPE (cloud deployments)**
      - **![#f03c15](https://placehold.it/15/f03c15/000000?text=+)**  I have not figured yet how to make `kubectl` command work while running Litmus docker container
        locally. Since Litmus is using `kubectl`, currently it is not possible running IDPE tests locally using
        Litmus docker image when test cluster is deployed in the cloud  
        
    **Running on IDPE (dev environments)**
      - >docker run --rm -e ETCD_HOST=http://host.docker.internal:2379 -e GATEWAY_HOST=http://host.docker.internal:9999 
      -e QUERYD_HOST=http://host.docker.internal:8093 -e TASKS_HOST=http://host.docker.internal:8276 -e TRANSPILERDE_HOST=http://host.docker.internal:8098 -e NAMESPACE=twodotoh 
      -e STORAGE_HOST=http://host.docker.internal:6060 -e STORAGE_CONTROLLER_HOST=http://host.docker.internal:6060 -e ONE_TEST/TEST_LIST=<path_to_test_file_or_test_list> 
      -v <local_dir>/:/Litmus/result quay.io/influxdb/litmus:<tag\>
      - **![#f03c15](https://placehold.it/15/f03c15/000000?text=+)** Smoke Tests need to be updated as well as running `kubectl` command is not working yet.
  
    **Running in Staging / Production Environments from Jenkins**
      - >docker run --rm -e GATEWAY_HOST=\<staging or production gateway url\> -e STAGING=--staging -e ADMIN_TOKEN=\<token\> -e ONE_TEST=src/cloud/rest_api/smoke/test_smoke_integration.py 
      -v \<local_dir\>:/Litmus/result quay.io/influxdb/litmus:\<tag\>
  
- **How to run Litmus from local repository**  
    - Prerequisites for running test(s) from local repo:
      - You need to be in `idpe/litmus` dir.
      - Copy `litmus_run_master.py` script from `idpe/litmus/scripts` directory to `idpe/litmus` directory.
      - Test(s) must be run from `idpe/litmus` directory.
      - `influxd` binary needs to be available (either built or downloaded from AWS S3) or, test cluster needs to be deployed locally (dev env) or in the cloud.  
      
  **Running on Open Source Platform** 
  - Running One Test using `inluxd` binary:
    - >python litmus_run_master.py --platform --gateway http://localhost:9999 --boltpath <path_to/influxd.bolt> --enginepath <path_to_engine> --binarypath <path_to_binary/influxd> 
    --tests <path_to_test_file>  
  - Running List Of Tests using `influxd` binary:
    - >python litmus_run_master.py --platform --gateway http://localhost:9999 --boltpath <path_to/influxd.bolt> --enginepath <path_to_engine> --binarypath <path_to_binary/influxd> 
    --tests-list <path_to_test_list_file>  
  - Running One Test using `influxd` docker image:
    - >python litmus_run_master.py --platform --gateway http://localhost:9999 --dockerimage --tests <path_to_test_file>  
  - Running List Of Tests using `influxd` docker image:  
    - >python litmus_run_master.py --platform --gateway http://localhost:9999 --dockerimage --tests-list <path_to_test_list>  
    
  **Running on IDPE (local dev environments)**  
  - Running One Test:
    - >python litmus_run_master.py --gateway http://localhost:9999 --queryd http://localhost:8093 --etcd http://localhost:2379 --transpilerde http://localhost:8098 
    --tasks http://localhost:8276 --storage_controller http://localhost:606 --storage http://localhost:6060 --kubeconf <path_to_config_file> --tests <path_to_test_file> 
    --namespace twodotoh  
  - Running List Of Tests:
    - >python litmus_run_master.py --gateway http://localhost:9999 --queryd http://localhost:8093 --etcd http://localhost:2379 --transpilerde http://localhost:8098 
    --tasks http://localhost:8276 --storage_controller http://localhost:606 --storage http://localhost:6060 --kubeconf <path_to_config_file> 
    --tests-list <path_to_test_list> --namespace twodotoh  
    
  **Running on IDPE (cloud deployments)**
  - Running One Test:
    - >python litmus_run_master.py --etcd http://etcd.<cluster_name>.svc.cluster.local:2379 --tasks http://tasks.<cluster_name>.svc.cluster.local:8276 
    --gateway http://gateway.<cluster_name>.svc.cluster.local:9999 --queryd http://queryd.<cluster_name>.svc.cluster.local:8093 
    --transpilerde http://transpilerde.<cluster_name>.svc.cluster.local:8098 --namespace <cluster_name> --storage http://storage.<cluster_name>.svc.cluster.local:6060 
    --storage_controller http://storage-controller.<cluster_name>.svc.cluster.local:6060 --kubecluster auth0@acceptance --kubeconf <path_to_config_file> 
    --tests <path_to_test_file>
  - Running List Of Test:
    - >python litmus_run_master.py --etcd http://etcd.<cluster_name>.svc.cluster.local:2379 --tasks http://tasks.<cluster_name>.svc.cluster.local:8276 
    --gateway http://gateway.<cluster_name>.svc.cluster.local:9999 --queryd http://queryd.<cluster_name>.svc.cluster.local:8093
    --transpilerde http://transpilerde.<cluster_name>.svc.cluster.local:8098 --namespace <cluster_name> --storage http://storage.<cluster_name>.svc.cluster.local:6060
    --storage_controller http://storage-controller.<cluster_name>.svc.cluster.local:6060 --kubecluster auth0@acceptance --kubeconf <path_to_config_file> 
    --tests-list <path_to_test_list>

  **Running in staging or production environments**
  - Running One Test:
    - >python litmus_run_master.py --gateway https://twodotoh.a.influxcloud.net --staging --admin-token \<admin token\> --org-name \<org name\> --org-id \<org id\> --tests src/cloud/rest_api/smoke/test_smoke_integration.py
  - Running List Of Tests:
    - >python litmus_run_master.py --gateway https://influxdb.aws.influxdata.io --staging --admin-token \<admin token\> --org-name \<org name\> --org-id \<org id\> --tests-list <path_to_test_list>
    
  
## **Deployment/Installations**
- **Running cloud 2.0 Open Source:**
   - Clone `influxdb` and `idpe/litmus` repos, in case You want to run `Litmus` from a local repo. Please refer to `README` files in there repos for any build questions.
   - You can either build `influxd` and `influx` binaries or download them from S3:
     - Building `influxd` and `influx` binaries:
       1. go to **influxdb** directory (make sure you pulled the latest commits)
          - checkout the branch you want to test against.
       2. run **make build** (you can run `make clean` before building binaries.)
          - build creates binaries in `influxdb/bin/darwin` location on MAC.
     - Downloading binaries from S3 and un-tar in location of your choice:
       1. Download **nightly** binary for MAC/Litmus:
          >https://s3.console.aws.amazon.com/s3/buckets/dl.influxdata.com/platform/nightlies/?region=us-east-1&tab=overview
       2. Download **alpha** release(s):
          >https://console.aws.amazon.com/s3/buckets/dl.influxdata.com/influxdb/releases/?region=us-east-1&tab=overview&prefixSearch=influxdb_2.0.0
   - Using **docker** image for open source nightly builds:
     >quay.io/influxdb/influx:nightly  
     
- **Running cloud 2.0 Development environments**
  - Install `kubone`, see [Kubone Documentation](http://docs.internal/development/kubone/)  
- **Running 2.0 in the cloud**
  - Using [fenvd](https://github.com/influxdata/idpe/tree/master/apps/fenv) running in the cloud:
    - >docker run --entrypoint /ctl/fenvctl --rm quay.io/influxdb/fenvd --host http://fenvd.fenv.svc.cluster.local new "<cluster_name>"  
  - Using `fenvd` running locally (useful for testing fenvd changes):
    - build `fenvd`: from `idpe/apps/fenv` run **`make build`**. It will produce the `fenvd` binary.
    - start `fenvd` server by running:
      - > ./fenvd --registry-email "user_email" --registry-password "user_password" --registry-username "user_name"
    - build `fenvctl` by running **`make ctl`** It will produce the `fenvctl` binary.
    - deploy test cluster locally:
      - >./fenvctl-linux-amd64 --host="http://127.0.0.1:8787" new "<cluster_name>"