Test Case Development guide

Introduction

Any given product is made of one or more features/services which are again a combination of multiple functionalities. So it is important to test at each level i.e functionalities, features and product. For testing each of these level, VTP has modelled them as below

• For testing a given functionality, its modelled as set of Testcase with required inputs (modelled as Parameters) and outputs (modelled as Results). Test case implementation is modelled as script.
• As service is made of set of functionality, for testing a given service, it is modelled as Testsuite. So Testsuite is made of set of testcases. VTP recommends to name the testsuite in a.b.c for hierarchy of test suites.
• As product is made of set of services, for testing a given product, it is modelled as scenario. So Scenario is made of set of testsuites and can be used to model a given testing scenario logically as well.

These model details are captured together into YAML file along with required scripts.

How to Design testing.

Pre-requisites

Consider new test suite name as 'compliance', run below command to create it

cp -R feature1 compliance
sed -i 's/feature1/compliance/g' compliance/build.sh
mv compliance/tc/open-cli-schema/testcase11.yaml  compliance/tc/open-cli-schema/compliance-tc-01.yaml
sed -i 's/feature1/compliance/g' compliance/tc/open-cli-schema/compliance-tc-01.yaml

Use 'compliance/tc/open-cli-schema/compliance-tc-01.yaml' as reference to create YAML for each of the test cases. And set the following fields in this YAML:

1. info.author - Author of the test cases

2. info.product - Scenario name

Design test cases

At first, decide the test cases at very granular level and create YAML for each of the test cases identified by referring the YAML created in pre-requisites. The set the following fields:

1. name - Name of the test cases

2. description - Test case description

3. parameters - List of Parameters with unique name, unique options, default value, etc

4. results - If test case produce results, add these sections, otherwise ignore.

4.1. results.attributes - List of output attributes with unique name, descripton etc

4.2 cmd.result_map - For each attributes defined above, fill it with JSONPath of of test case output generated by test scripts. so test script should produce the output in json.

5. cmd.command - Test script and its mapping of arguments from the parameters defined in this yaml. Always write the script in this form '$s{env:OPEN_CLI_HOME}/script/'' , where script-name is relative path of test script kept under tc/script. For example, if script tc/script/f1/s1.sh then this entry should be ''$s{env:OPEN_CLI_HOME}/script/f1/s1.sh'

Test case implementation

For every test case designed above, implement required scripts under tc/script folder. And this script will read input arguments and execute the required testing and produce the output in JSON. By default its read from stdout console. But if output is required to write into json file, the VTP mandates to take that output file name as one more arguments to the scripts. so following additional changes required in yaml

1. cmd.command: ... $s{tmp:result.json}

2. cmd.output: $s{tmp:result.json}

NOTE: additional argument name and output name should match.

NOTES

In this release, only one tester is supported in edgeT so its recommended to keep all testsuites and test cases under one folder like 'feature1'.

To build docker

Run below script to build the docker image, it will be build as edgegallery/edget-feature1-testcase , where feature1 will be replaced with actual test name.

build.sh