Metadata-Version: 2.1
Name: regressionstest-pipeline-dev
Version: 0.1.2ubuntu1-1726569791.gbp996e6d
Summary: Includes Regressionstest and further testing tools
Home-page: https://gitlab.consolinno-it.de/leafletfirmware/testing/regressiontest-pipeline-dev
Author: Moritz kolb
Author-email: m.kolb@consolinno.de
License: UNKNOWN
Project-URL: homepage, https://www.consolinno.de
Platform: UNKNOWN
Classifier: Programming Language :: Python :: 3
Classifier: Operating System :: OS Independent

# Regressionstest-pipeline-dev

This repository contains all regression tests of the 1U0022-CO product of Consolinno-Energy.

It consists of several component tests. These tests are executed via the GitLab pipeline or locally.

## Local

First you have to create a .env in the current folder to add the secret variables: For the ConfigCheck: `CI_kaco_key`, `CI_conems_key`. For the Konfigmanager API: `CI_bearer_token`. For the Confluence API: `CI_conflu_token`, `CI_conflu_user`.
You can find the values for the secret Variables in the Gitlab Repo Settings / CI/CD / Variables. For the Confluence API you should use your credentials. Setting up your API Token follow [this Introduction](https://docs.searchunify.com/Content/Content-Sources/Atlassian-Jira-Confluence-Authentication-Create-API-Token.htm).

    CI_conflu_token = ***
    CI_conflu_user = ***
    CI_bearer_token = ***

    CI_conems_key = ***
    CI_kaco_key = ***

To run the tests on your local machine, please first modify the `Regressiontest-config.ini` file:
You need to set your local SSH private key path if it is different from `/home/<username>/.ssh/id_rsa`.
At `[testdevice]` you have to setup the testdevice characteristics. If the Flashtype is Mender, your have to setup
bowth `INFOFILE` types. Be soure bowth *system.info files are present in the project. If `FLASHTYPE` is inititial just setup the `COMPAIR_INFOFILE`.
Also check the setting for the individual test components.

Then, add your public SSH key to the `authorized_keys` on the test device via `mender_terminal` with `nano` or `vi`.

    vi .ssh/authorized_keys

Next, run the `execute_test_locally.py` script to start the Testsoftware.

Example:

    python3 -m src.regression_test.main
    python3 -m src.wp_test.main
    python3 -m src.plim_test.main
    python3 -m src.nymea_test.main

## Pipeline

Pipeline jobs are in progress

## Components

The regression tests cover several components. Each component test runs on its own, but they all use the same configuration file and document the results in the same `outputfile.json`.

### ConfigCheck

The ConfigCheck Tests if the Configuration which was created for the device is correct.
For more information about the Testcases, check out: [ConfigCheck](https://consolinno.atlassian.net/wiki/spaces/~62f39a8532850ea2a3268713/pages/531497005/ConfigCheck)

#### Configurationfile: testconfig.json

The Test is configured by the Test config files according to the Config manager Version.
The Test config files consist of reference files, and a `testconfig.json`.
The reference files present the valid content of specific configuration files. In the `testconfig.json` there is 
the Test information of each Config file to test.

The Project is using a Configfile in JSON-Format, holding informations about every file in the Config.

##### Structure of the `testconfig.json`

The Config file has a JSON-Objects with different key pairs. Here are the valid properties:

- **`textfiles` (Object):** contains all text files

  - **`<textfile name>` (Object):** contains the properties of the individual file
    - **`pattern_file` (String):** name of the corresponding reference file
    - **`indevid_lines` (Object):** present if there are individual lines per type or environment
      - **`<number>` (Object):** Line number of the individual line.
        - **`replace_pattern` (String):** string of placeholder
        - **`type` (Object):** lists the individual string per type
          - **`co` (String):** individual co string
          - **`base` (String):** individual base string
        - **`stage` (Objekt):** lists the individual string per environment
          - **`dev` (String):** individual dev string
          - **`qa` (String):** individual qa string
          - **`prod` (String):** individual prod string
        - **`serialnumber` (String):** indicates different types of serial numbers "upper_case", "lower_case" and "productnr"
  - **`present` (Object):** indicates where the file should be present
    - **`stage` (Array):** contains all stage options (dev,qa,prod)
    - **`type` (Array):** contains all type options (co,base)
  - **`covered` (bool):** is used to mark the file as tested during the test should always be false

- **`keyfiles` (Object):** Contains all key files
  - **`<keyfile name>` (Object):** contains the properties of the individual file
    - **`keylength` (int):** key length
    - **`keytype` (String):** there are different key types: (PRIVATE KEY, CERTIVICAATE,RSA PRIVATE KEY, PUBLIC KEY, ssh-rsa or root)
  - **`cryptotest` (Object):** only present if there is a key pair in the Dir
    - **`partner` (String):** name of the partner key
    - **`type` (String):** (private, public)
    - **`covered` (bool):** is used to mark if the test is done 
  - **`present` (Object):** indicates where the file should be present
    - **`stage` (Array):** contains all stage options (dev,qa,prod)
    - **`type` (Array):** contains all type options (co,base)
  - **`covered` (bool):** is used to mark the file as tested during the test should always be false

##### Example `testconfig.json`:

Here is a example for the `config.json`-file:

```json
"device_type": {
                "pattern_file": "devicetype-pattern.txt",
                "indevid_lines": {
                    "1": {
                        "replace_pattern": "DEVICETYPE",
                        "type": {
                            "co": "CO",
                            "base": "BASE"
                        }
                    },
                    "1.0": {
                        "replace_pattern": "PRODUCTNR",
                        "serialnumber": "productnr"
                    }
                },
                "present": {
                    "stage": [
                        "dev",
                        "qa",
                        "prod"
                    ],
                    "type": [
                        "co",
                        "base"
                    ]
                },
                "covered": false
            },
```

### Shell-Test

JUST LOCAL USE IS ACTIVATED FOR NOW!!!!

This test checks if the system is overall running correctly. For the test, a shell script is copied and executed on the test device.  
For more information, check out: [Shell-Test](https://consolinno.atlassian.net/wiki/spaces/~62f39a8532850ea2a3268713/pages/500959265/SHELL+Test)

### CRC-Config-Check

JUST LOCAL USE IS ACTIVATED FOR NOW!!!!

Compairs the content of all Configfiles from the Configmanager and the Leaflet

### Nymea-Test

#### Teststart

First, the network undergoes scanning. Devices with an open port 4444 are targeted. A "Hello" message is dispatched to these devices to retrieve their name and authentication status. If a device responds with "initialSetupRequired" set to "true," it indicates that the device is open, allowing the creation of a user and initiating authentication.

Subsequently, the test client establishes a connection with the device matching the provided serial number in the command-line variables. Either:

    1. There is no user already configured, so a new automated user is created.
    2. A user already exists, an attempt is made to authenticate using the automated user credentials.

The automated user setup is:
    username: appium
    password: Einfachso1

```plantuml

@startuml
    skinparam backgroundColor #EEEBDC
    actor Tester
    Tester -> "Testpc" : serial number
    "Testpc" -> "nymea" : JSONRPC.Hello
    "nymea" -> "Testpc" : initialSetupRequired: "true"
    "Testpc" -> "nymea" : JSONRPC.CreateUser
    "nymea" -> "Testpc" : ok
    ==Configtest==
    "Testpc" -> "nymea" : ModbusRtu.GetModbusRtuMasters
    "nymea" -> "Testpc" : modbus config
    "Testpc" -> "nymea" : Hems.GetHEMSVersion
    "nymea" -> "Testpc" : Hems vers.
    "Testpc" -> "nymea" : Configuration.GetConfigurations
    "nymea" -> "Testpc" : webserver
    "Testpc" -> "nymea" : Configuration.GetMqttServerConfigurations
    "nymea" -> "Testpc" : mqtt server config
    ==Powersumtest==

    ==EV-charachintest==
    ==Docu==
    "Testpc" -> Tester : Documentation

@enduml
```

future -> maybe option that a user already exists... 
Then the tests will begin:

#### Testcases

#### Configurationtest

With help of the first calls the configuration of nymea will be checked: server, modbusrtu...

#### Powersumtest

Simulated Things will be added, through the change of PV-inverter settings and wallbox the power out/input can be manipulated
the correct sum has to be calculated.
The Simulated Things will be added like this: getThingclasses - (all simulated thingclasses will be picked) -> DiscoverThings(reseive DescriptorId) -> check SetupMethodJustAdd!!! if true its AddThing if not its PairThing!!!
Inverter has to be set to max output so with the help of changing the power of the Thing there is a variable Power possible.
For the Wallbox, it has to be in the "Charge always" Charging Process. 

#### EV-Charging test

The different charging options will be tested here:
with help of the charging settings the different stats of the charging events will be walked through.
With help of the current status the correct workflow of the charging mode will be checked.


#### Development Process

1. Find correct leaflet
2. create a user and connect
3. add Things
4. config test
5. sum test
6. \todo charge test


## Confulence API

After all component tests have been run, a configtestoutput.json file is generated in which all test results are saved.
These test results are then loaded onto Confluence via `Confluence_report.py`. 
A template is used for this, whose ID and the ID of the parent document under which the file is to be saved must be entered under `Regressiontest-config.ini`. 
On the produced Confluencepage each Testcomponent will produce his own table with all the driven Testcases.
All Testcases with an Error will get a red colloring.
If there is a failed testcase that is already recorded in a ticket, add the test case name and the corresponding ticket number in the `Regressiontest-config.ini` under the test component in the `ERRORTICKETLIST` variable.

## Componenttests besites to the Regressiontest

In this Repo there are also Tests which will only used for single releaces. To test a new feature.

### Heatpump optimizer (wp_opti)

This Skript reads a given PV maximum curve:

    self.pv_test_data = {'x': [0,15,30,45,60,75,90,105],'y':[0.6,0.2,0.9,0.2,0.2,0.9,0.6,0.3]}

Whereby the x stands for time in minutes after starting the test and the y for the percentage of power surplus.
Acording to the y value the simulated solarpower is changed to set the correct value.
Then the actual Power surplus and the current SG-Ready state is written to a .csv file, witch is exported in the end.

### Plim_test

The Plim Test uses the W2 and W3 Relays of the tested Leaflet device, in order to run this Test there is some Wiring to be done.

5V - 31, GND - 32, 1 - 34
5V - 41, GND - 42, 2 - 44

The Test will Start a Charging session at the beginning and then Simulate all 4 possible States the Plim con have and checks if the HEMS is reacting the correct way.

The outcome of the test is saved to a .json file and uploaded to confluence,
so don't forget to setup the `Regressiontest-config.ini` in order to login to
the device and upload stuff to confluence.


### deleate Childrenpages in Confluence

`Confluence_report` can also be used to delete all child pages of a confluence page. The -dc (deleate children)option is specified for this it also needs the parent page.
The `SPACE_KEY` is setup in the `Regressiontest-config.ini`.

    python3 -m src.report.Confluence_report -dc 589342523


