Claus Topholt
Selenium is a great tool for professional programmers who want to automate web application testing, but for users who don’t write code nor want to, there are better options.
This post compares the code-based Selenium web automation to Leapwork web automation.
What is Selenium?
Selenium WebDriver is the world’s most popular open-source framework for automating browsers.
Its API can be used to drive any action that takes place in a browser natively, meaning that it can click on buttons and type text into fields, just as a real user would.
Selenium has lots of benefits:
- Selenium is open source, free to use, and free of charge
- Highly extensible
- Works across different browsers
- Supports various operating systems
- Supports mobile devices
- Can run while the browser is minimized
- Can run in parallel
Professional programmers with years of experience in writing reusable and maintainable code could find Selenium a great solution. Especially if you're only automating web applications in a browser.
It has a well-designed programming model and a large ecosystem of pay-as-you-go cloud services such as BrowserStack and Sauce Labs to run in parallel across multiple devices and browsers.
However, if you are a tester or you are managing a team of QA professionals, using Selenium may cause more problems than it solves; it’s a low-level solution to a high-level problem.
Low-level means dealing with all the tiny, intricate technical details.
Test automation, on the other hand, requires a high-level approach that hides complexity from the user through abstraction.
In the end, it takes an enormous effort to implement, and more often than not, Selenium code grows into a beast that is impossible to maintain.
Using Selenium for a simple test automation example
Let’s start by taking a look at a simple test automation example:
- Open a Chrome browser and navigate to amazon.com
- Type “screwdriver” in the search field on the main page
- Click the “screwdriver set” suggestion link
- Validate that an image of a screwdriver set appears on the next page
- Close the browser
We’ll see how to code it for Selenium WebDriver and then afterwards what the same example looks like in Leapwork, which uses Selenium WebDriver under the hood.
First, here’s what the Selenium example looks like:
An example of Selenium automation, written with C# code.
As explained in the Sauce Labs’ introduction to Selenium, there are seven basic steps in creating a Selenium script for test automation:
- Create a WebDriver instance
- Navigate to a web page
- Locate an HTML element on the web page
- Perform an action on an HTML element
- Anticipate the browser response to the action
- Run tests and record test results using a test framework
- Conclude the test
It sounds straightforward, but it’s clear from the example above that it’s a very complicated and time-consuming task to write the appropriate code.
It takes an experienced programmer about 30 minutes to solve this small example, and even if the programmer is highly skilled, it would still require several trips to Google to look up how to do things correctly.
But the biggest problem facing non-professional programmers isn’t actually getting the automation code to work initially.
It’s maintenance of the scripts, particularly in an enterprise environment. Below is an example of an error that might appear periodically after the script has been running fine for a few weeks:
Related reading: What is Enterprise Application Testing?
An example of an error message that might suddenly show when running Selenium automation. Users might not even know what the error message means.
How to deal with that? Who to call? Google results point to some articles from 2011, which looks very outdated…
In contrast, here’s a screenshot of the same test automation example built in less than two minutes with Leapwork.
It’s, doesn’t require any programming skills and can be easily maintained.
The same test case automated with Leapwork, made up of six standard components.
Video demonstration: Selenium vs. Leapwork
We’ve asked two testers to build the same test case with their respective approach: Code-based automation using Selenium and codeless UI automation using Leapwork.
We timed the efforts to evaluate which of the two approaches is more efficient. Spoiler alert: It’s not even close.
Selenium comparison: the pros and cons of using Selenium vs Leapwork
We’ve put together a list of pros and cons of using Selenium WebDriver directly with code versus using Leapwork to perform the same tasks.
You can also download a PDF version of the comparison here.
Usability and adoption |
||
|
|
Using Selenium WebDriver with hand-written code |
Using Leapwork, which uses Selenium under the hood |
|
Programming needed |
Can only be used by programmers, who get low-level access to browser behavior and actions through using any programming language (C#, Java, Python, PHP, etc). |
Can do the exact same thing with easy-to-use building blocks, and without having to read or write a single line of code. Perfect for non-technical users – or anyone who just doesn’t want to spend time on programming. |
|
Enterprise support |
No enterprise support exists, the Selenium project is run by the open source community. |
On-demand live support from automation specialists. |
|
Data-driven automation |
Data-driven automation requires sophisticated programming and separate management and storage of data assets. |
Data-driven automation is included out of the box. It is done by visually connecting data sources such as databases, API endpoints and Excel files with other building blocks. |
|
Finding web elements |
Uses highly complex CSS or XPath “selectors”, which require deep technical knowledge. |
Point and click to capture any web element. Smart algorithms then automatically find the best way to locate it again in the future. |
|
Road-map influence |
Although Selenium is open source, the only way to influence the road map is to actively participate in the community programming effort for Selenium itself. |
All enterprise users have direct access to live chat and product team representatives. About a third of all product development is a result of this dialogue. |
|
Training materials |
A good ecosystem and lots of code samples exist on the internet for professional programmers to use. Some tutorials on YouTube and many professional courses exist but require technical skills. |
All users have access to a comprehensive learning center with many hours of free videos. Enterprise users can participate in a professional certification program. |
|
Speed of adoption |
Typically takes 3-6 months for an initial project, then another 6-12 months for framework programming. |
Most users are able to adopt within 3-5 days, using a structured learning and on-boarding program. |
Documentation and governance |
||
|
Selenium WebDriver |
Leapwork |
|
|
Reporting and dashboards |
No reporting or dashboard solution is included. Programmers can custom-build their own reporting framework (typically takes 3-6 months). |
Reporting and dashboards are included out of the box. Results can also be pushed to other tools such as JIRA. |
|
Automatic video recording |
Video recording is not included and is very difficult to achieve, even with a custom-built solution. Programmers are typically limited to saving screenshots with no retention policy. |
Everything is automatically recorded on video, and a retention policy can ensure old videos are automatically deleted. |
|
Object repository and version history |
An object repository is not included, but programmers can custom-build their own and decide on implementing code patterns such as Page Object Model. Version history can be achieved using source control systems such as Git, which most non-technical users find impossible to navigate. |
All assets such as web element locators and flows are structured and stored in an encrypted database with built-in version history. |
|
Access control |
No access control is included and is difficult to achieve with a custom-built solution. |
Access control, both locally in the encrypted database and through Active Directory is included. |
|
Tamper-proof audit trails |
No audit trail capability is included and is difficult to achieve with a custom-built solution. |
All data is stored in an encrypted database with no direct access. Even administrators cannot change the audit log. |
Applications Supported |
||
|
Selenium WebDriver |
Leapwork |
|
|
SAP GUI |
SAP GUI is not supported. Selenium only supports web applications and it’s not possible to access elements in desktop applications outside the browser. |
Works across all technologies. SAP GUI is natively supported. |
|
Windows applications |
Windows applications outside browsers are not supported. |
Works across all technologies. All Windows technologies such as WinForms and WPF are natively supported. |
|
Citrix applications |
Citrix applications are not supported. |
Works across all technologies. Citrix and other virtualization technologies are supported with advanced text and image recognition capabilities. |
|
Mainframe applications |
Mainframe terminal applications are not supported. |
Works across all technologies. Mainframe terminal applications are supported with advanced text and image recognition capabilities. |
|
Java, IBM, and Oracle applications |
Java applications, including IBM and Oracle Forms are not supported. |
Works across all technologies. Java, IBM, and Oracle Forms applications are natively supported, including the older Java 1.6 standard. |
|
Native mobile and web |
Requires the use of Appium |
Test on any combination of device, OS and browser. Spin up your test in the Cloud with our HeadSpin integration. |
Test execution and integrations |
||
|
Selenium WebDriver |
Leapwork |
|
|
Scheduling |
No scheduler is included. Programmers can use a unit-test framework in combination with a custom-built scheduler or build pipeline. |
Scheduling and ad-hoc running is included out of the box. Flows can also be triggered from a build pipeline or other third-party system. |
|
Parallelization |
Can run in distributed, parallel environments, but requires programmers to build a custom automation framework (typically takes 6-9 months). |
Can run in distributed, parallel environments out of the box, including on BrowserStack, Sauce Labs and Selenium Grid as well as on Leapwork agents, where web, desktop and virtual applications can be automated together. |
|
Built-in REST API |
No REST API is included, but the WebDriver wire protocol is open source, and custom-built REST APIs can be made by programmers. |
A full-fledged REST API is included out of the box, making it easy for DevOps to integrate with any third-party system. |
|
Built-in CI/CD plugins |
Any CI/CD platforms can be used by custom-built solutions. A good ecosystem and lots of code samples exist on the internet for professional programmers to use. |
All popular CI/CD platforms can easily be integrated by DevOps using plugins or the REST API. |
The complete guide to automating tests with Selenium
You can access the full pdf version of our guide to learn more about Selenium and how it stacks up against Leapwork. It covers the skills, effort and resources needed to use Selenium to its fullest. And, it proposes a web automation solution that doesn't require technical programming skills.
