[REC-22] Create CD testrunner Created: 27/Jun/19  Updated: 04/Sep/19  Resolved: 04/Sep/19

Status: Done
Project: Radio Edge Cloud
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: New Feature Priority: Highest
Reporter: Paul Carver Assignee: Amin M
Resolution: Done Votes: 0
Labels: None
Σ Remaining Estimate: Not Specified Remaining Estimate: Not Specified
Σ Time Spent: Not Specified Time Spent: Not Specified
Σ Original Estimate: Not Specified Original Estimate: Not Specified

Sub-Tasks:
Key
Summary
Type
Status
Assignee
REC-23 Create initial structure for CD testr... Sub-task In Review kinkwan tsui  
REC-24 Create JJB for ci of the cloudtaf rep... Sub-task Done  

 Description   

Create the necessary code that can be called by contributor CD systems (such as the one already run by AT&T or by CD systems that may be setup in the future by any new REC contributors.)This story does not include actually writing any tests, but it does include writing some documentation in a new child page under https://wiki.akraino.org/display/AK/Radio+Edge+Cloud+Developer+Documentation that explains how developers can add tests such that they will be automatically run the next time an LF Jenkins job is triggered (e.g. when a new change is posted to https://gerrit.akraino.org (verify job) or when an existing change on Gerrit is merged (merge job)) and how new contributors can invoke the top level testrunner in a CD system that they setup in the future.

It was generally agreed by people in a preliminary discussion that we want to standardize on Robot Framework as the top level testrunner. If it looks like it is necessary for Robot to invoke some other tooling, that should be brought back to the team for discussion.

The code to be written needs to have the following characteristics:

  • Can be run on a deployed REC cluster
    • The expected usage is that Jenkins will transfer the test runner to a newly deployed REC cluster and execute it on the cluster
  • Clones one or more git repositories containing all the test case code (exact location TBD, but it'll be on https://gerrit.akraino.org with the other TA/REC repos). The reason for the "or more" is that we may want to put tests that apply to all Telco Appliance blueprint family members into a separate repo from tests that are specifically radio related.
  • Executes the tests in sequence
    • Stretch goal: think through whether it is possible to architect the test runner to allow for some way of parallelizing tests
  • Logs parse-able test results so that we can have some sort of post-processing of test results in the future.
  • Supports a concept of non-fatal failure and/or experimental test case so that as new test cases are added we can have the test running execute them but not count them towards the overall pass/fail status
  • Returns an exit code to Jenkins to report whether the test suite was successful


 Comments   
Comment by Amin M [ 04/Sep/19 ]

Two separate builds have been created for two OE clusters in Middletown

http://jenkins2.mtlab.att-akraino.org/job/Cloudtaf_test_suite_OE1/

http://jenkins2.mtlab.att-akraino.org/job/Cloudtaf_test_suite_OE2/

Comment by Amin M [ 04/Sep/19 ]

http://jenkins2.mtlab.att-akraino.org/job/Cloudtaf_test_suite/ being used as testrunner to invoke robot cases as a jenkins job.

Generated at Sat Feb 10 06:04:28 UTC 2024 using Jira 9.4.5#940005-sha1:e3094934eac4fd8653cf39da58f39364fb9cc7c1.