Microsoft ADO Test Plan Integration

Introduction

In ADO, it is a tedious task to run test cases manually in every release. When you are creating test cases in ADO, you can maintain it using a test plan structure or individually create work items and manually try to execute those test cases.
This integration allows you to automate test cases in Leapwork. The automated test cases can be executed multiple times. You can analyze execution report and view results on ADO.

This integration allows you to:

• Setup connection with ADO.
• Import ADO test cases in Leapwork flow.
• Sync ADO test cases in Leapwork flow.
• Import ADO test plan structure in Leapwork runlist.
• Publish execution results on ADO.
• Create bugs from Leapwork to ADO.

Import ADO test cases in Leapwork

Once ADO connection is created, we can import flows as part of test cases in Leapwork. 

1. Right click in the left window and select Import from ADO.
2. Select a project and click Import.
   Note: Only test cases marked as Planned in ADO, under the Automation Status field, will be imported.
3. The test cases are imported. Click OK.
4. On left, the test cases are created in project folder Leapwork-ado-sbx. Within the project folder, the structure to be followed will be defined by the area in ADO.
5. Click to expand a project folder. The ADO imported test cases list appears. The test case filename includes testcase id and test case name. Once all the test cases are imported, you can record the flows and run test cases in run list. In Flow settings, the test case link and summary appears. 

Sync ADO test cases in Leapwork flow

Important: The option to sync ADO test cases is available starting from release 2024.1.486. If you are using an earlier release, this feature will not be available. Please ensure your Leapwork version is up to date to access this functionality.

If a project is ready to be synced, a banner will appear at the top of the flows explorer/list.

1. Click Review button.
   
2. Select a project. All projects associated with the ADO sync appear.
   
3. Click Sync.
   Note: Only test cases marked as Planned or Automated in ADO, under the Automation Status field, will be synced.
4. Once the sync finishes, a confirmation pop-up window appears. Click Ok. 
   
   

Important notes when syncing

1. Deleted Test Cases in Azure DevOps: If a test case or project is accidentally deleted in ADO and you hit the Sync button in Leapwork, the corresponding test cases will not be deleted in Leapwork. This protects your existing automation flows.
2. Test Case Status Changes: 
   1. The status of the flow in Leapwork will be updated to Retire if the test case has been permanently deleted or moved to Not Automated in ADO during synchronization.
   2. If the test case is later moved back to Planned, the flow in Leapwork will be updated from Retire to Draft during the next sync.
      
3. New Test Cases Added in ADO: If new test cases are added in Azure DevOps after the initial import, they will not automatically be fetched during synchronization.
   
   1. The Sync functionality updates existing flows and test cases.
   2. New test cases can be imported using the Re-import functionality in Leapwork.

Import ADO test plan in Leapwork

1. In Run lists, right click and select Import from ADO.

2. Select a project. All the test plans related to that project appears. Each test plan is created based on the test suite and configuration.  Select a test plan and click Import.
   Note: Only test points marked as Planned or Automated in ADO, under the Automation Status field, will be imported.

3. Click View Details. If there is no test case in a test plan, a message appears - Automated test cases are not available in ADO.
   The information displayed in the ADO Import pop-up window is dynamic based on the result of the importing process:
   • TC Successfully Imported: Imported all suitable test points successfully.
   • Automated TC Not Available: It displays the test cases not imported due to unavailability in ADO or Leapwork.
   • Collision: It displays scenarios where Leapwork already has manually created run lists or folders with the same names in the run list section.

4. Click OK. On the left, run list and configurations appears. Each run list has test cases that you imported in previous steps. Runlist is created on the basis of configuration (Windows 10/11).
5. Right click on a runlist and select New schedule.

6. Enter title of the schedule and click Save. The new schedule is created on the left.

7. Right click on the new schedule and select Run Now.

Publish execution results on ADO

In ADO, you can verify that while execution, test run is in progress or completed as shown below.

1. In ADO, click refresh.
2. Click on a test run.
3. Click Run summary. On right in pie chart, you can see that the result shows that one test case passed and one failed.

4. Click Test Results. You can see that the result shows that one test case passed and one failed.
   Note: Once a test case is run, the status of that test case will be automatically change from Planned to Automated in ADO, under the Automation Status field.

5. Click Progress Report. You can analyze results based on test suits and available filters.

Create bugs from Leapwork to ADO

You can create bugs directly in Leapwork; these bugs will then be reflected in ADO.

1. In Leapwork, click Reporting.
2. Right click on a specific flow with Failure Status.
   
3. Select Create a bug in ADO.
4. On the Create a bug in ADO pop-up window:
   1. Enter the title of the bug in the Title field.
   2. Click on the Priority drop-down and select a priority from 1 to 4.
      

      Note: The default option from ADO is 2.

   3. Click on the Severity drop-down and select a severity from 1 (Critical) to 4 (Low). 
      

      Note: The default option from ADO is 3.

   4. Enter the build number or version of the bug in the Found in build field.
   5. Give the reproduction steps in the Repro Steps field. 
5. Click Create.