Differences between Web, Desktop UI, and Virtual Desktop Automation

Leapwork supports advanced automation of desktop applications, web applications and applications running in a virtual desktop (Citrix, Remote Desktop, terminals, etc.). The different automation needs are enabled by different technologies.

This article outlines the differences between the three main automation types: Desktop UI, web, and virtual desktop automation.

Building blocks

All building blocks are available at all time during the design of a flow, but it's important to use the right building block at the right time. In the following some rules of thumb for different application types are described.

Desktop UI:  

This category contains building block primarily used for desktop applications. This includes any Office product (Outlook, Excel, Word), Winforms application, Java desktop applications, SAP etc.

The desktop UI blocks can also be used to automate web applications running in IE and Edge browsers, which can be necessary due to security restrictions.

Virtual Desktop:

The "Image and Text recognition" and "Mouse and keyboard" building block categories contains the main building blocks used for Virtual Desktop automation. This covers Citrix application, remote desktop, Mainframe interfaces, terminals etc. 

The building blocks in the "Image and Text recognition" category are obviously using image and text recognition, which often requires a slightly bigger setup in order to support both flow design and hardening of the flows. So only use building blocks form this category if it is necessary.

Learn more about the best practices for Virtual Desktop automation: Best Practices for Automating with Image & Text recognition

Web: 

The building blocks in this category covers all web sites and applications, and it is typically not needed to use any building blocks from the previous described categories (Desktop UI, Image & Text recognition) to automate the applications.

Environments

The building blocks exclusive to the Virtual Desktop cases ("Image & Text recognition" and "Mouse And Keyboard") are using Image & Text recognition and are hooking into the mouse and keyboard driver on the machine where the text case is executed, to drive the execution of the automation flow forward. In order to do so, the Leapwork Agent needs to be installed on the machine where the case is running. When an automation flow is executed, the Leapwork Agent continuously streams a video of what is happening on the screen back to the Controller. The Leapwork Agent inspects what is happening on the screen and uses this to drive the automation flow forward. So if you are executing Virtual Desktop cases these can only work on Environments of type "Remote (Agent)".

This is also true for the Desktop UI cases, which are also executed via the Leapwork Agent. In these cases, using the building blocks from the Desktop UI category, the Leapwork Agent is using Window UI Automation Framework to interact with both the operative system (Windows) and any desktop application based on Microsoft frameworks (MFE, .Net, WPF etc.). Any Java-based application is supported in a similar way. As for Virtual Desktop cases, the agent is streaming back the result and a video representation of the executing case to the Controller.

The Web Application blocks are using Selenium as the engine and they generate Selenium code on the fly when the test cases are executed. This means that a web case can be executed on all Environments that support/are based on Selenium, which includes the Leapwork Agent as well. If you are executing Web cases it can work on all types of the environment (currently: Remote, Selenium Grid, Sauce Labs, Browser Stack). 

Schedules

When you create schedules you will typically start by selecting a number of test cases to include in the schedule. Depending on the selected automation flows you will only be allowed to select environments that fits all the selected flows.

For example, if you have selected one Desktop flow and three Web flows, the Desktop flow will limit the environments available to only the "Remote (Agent)" type.

Go to Learning Center.