Claus Topholt
Selenium IDE is a great open source tool for HTML-savvy users who want record and playback automation of simple web applications, but it isn’t suitable for enterprises and has very limited functionality. This post compares Selenium IDE with Leapwork enterprise web automation.
Selenium IDE is an open source record-and-playback test automation tool for simple web applications. It uses Selenium WebDriver, which is the world’s most popular open-source framework for automating browsers. Selenium test automation can be used to drive any action that takes place in a browser natively, meaning that it can click on buttons and type text into fields, just like a real user would.
While Selenium WebDriver is an excellent code framework for professional programmers to automate enterprise web applications, Selenium IDE is more limited in scope. It’s targeted at HTML-savvy users who need a fast way to record and playback automation of simple web applications, and works as a browser extension in Chrome and Firefox. A previous version was designed only for Firefox.
Apart from the browser extension, it’s possible to playback any recorded test in parallel using a command-line interface:
Using a command-line interface to playback recorded tests with Selenium IDE.
Limitations of Selenium IDE
If you are a tester or you are managing a team of QA professionals, Selenium IDE may sound like a good way to get started with test automation, but you will quickly find yourself at a dead end for many reasons, such as:
- The tool offers very limited functionality and doesn’t work on all types of web applications. For example, at the time of writing, it is not possible to automate amazon.com in Chrome.
- Record and playback tests are quite fragile and prone to breaking whenever the user interface changes, and are notoriously hard to maintain.
- It’s very hard to reuse partial recordings across multiple tests. When you try to record a sequence of actions that you perform on a website or a desktop application, most recorders will record way too many actions, and typically, the user doesn’t have a clear understanding of why.
- Driving test automation with data from data sources such as Excel files or databases is not possible.
- Although the tool is technically code-free, it requires a strong understanding of how Selenium WebDriver code works, and of course HTML itself.
From an enterprise testing perspective, it’s also a significant concern that the tool comes with no enterprise capabilities and no support SLA; There are no built-in (or even possible) collaborative, user management, audit trail, scheduling or reporting functionalities. And there’s no-one to call when it stops working.
Related reading: What is Enterprise Application Testing?
But the biggest problem facing non-professional programmers isn’t actually getting the automation code to work initially -- it’s maintenance of the scripts, particularly in an enterprise environment, that are causing biggest headaches. Learn more about how to reduce the maintenance workload with codeless test automation.
In comparison, the following is a simple test automation case that searches amazon.com for “screwdriver”, clicks the “screwdriver set” suggestion and then validates that an image of a screwdriver set appears, built in less than 2 minutes with Leapwork. It’s completely visual, doesn’t require any programming skills and can be easily maintained:
A test case automated with Leapwork, made up of six standard components.
The Pros and Cons of using Selenium IDE vs Leapwork
We’ve put together a list of pros and cons of using Selenium WebDriver directly with code versus using Leapwork to perform the same tasks.
You can also download a comparison of test automation tools here.
Usability and Adoption |
||
|
|
Using Selenium IDE record and playback |
Using Leapwork, which uses Selenium under the hood |
|
Finding web elements |
Uses highly complex CSS or XPath “selectors”, which require deep technical knowledge. |
Point and click to capture any web element. Smart algorithms then automatically find the best way to locate it again in the future. |
|
Technical expertise required |
Can only be used by HTML-savvy technical experts, due to the way locators, browser behaviours and actions are used. |
Has easy-to-use building blocks, and without having to read or write a single line of code. Perfect for non-technical users – or anyone who just don’t want to spend time on programming. |
|
Reusable modules |
Not possible to create reusable components or modules. |
Reusable, modular sub-flows are easy to create and maintain. |
|
Data-driven automation |
Data-driven automation is not directly possible. It requires sophisticated programming and separate management and storage of data assets. |
Data-driven automation is included out of the box. It is done by visually connecting data sources such as databases, API endpoints and Excel files with other building blocks. |
|
Enterprise support |
No enterprise support exists, the Selenium project is run by the open source community. |
On-demand live support from automation specialists and enterprise consulting partners. |
|
Road-map influence |
Although Selenium IDE is open source, the only way to influence the roadmap is to actively participate in the community programming effort for Selenium itself. |
All enterprise users have direct access to live chat and product team representatives. About a third of all product development is a result of this dialogue. |
|
Training materials |
Some technical tutorials and documentation. |
All users have access to a comprehensive learning center with many hours of free videos. Enterprise users can participate in a professional certification program. |
Documentation and Governance |
||
|
|
Selenium IDE |
Leapwork |
|
Reporting and dashboards |
No reporting or dashboard solution is included. Programmers can custom-build their own reporting framework (typically takes 3-6 months). |
Reporting and dashboards are included out of the box. Results can also be pushed to other tools such as JIRA. |
|
Automatic video recording |
Video recording is not included and is very difficult to achieve, even with a custom-built solution. Programmers are typically limited to saving screenshots with no retention policy. |
Everything is automatically recorded on video, and a retention policy can ensure old videos are automatically deleted. |
|
Object repository and version history |
An object repository is not included, but programmers can custom-build their own and decide on implementing code patterns such as Page Object Model. Version history can be achieved using source control systems such as Git, which most non-technical users find impossible to navigate. |
All assets such as web element locators and flows are structured and stored in an encrypted database with built-in version history. |
|
Access control |
No access control (authentication and authorization) is included, and is difficult to achieve with a custom-built solution. |
Access control, both locally in the encrypted database and through Active Directory, is included. |
|
Tamper-proof audit trails |
No audit trail capability is included, and is difficult to achieve with a custom-built solution. |
All data is stored in an encrypted database with no direct access. Even administrators cannot change the audit log. |
Applications Supported |
||
|
|
Selenium IDE |
Leapwork |
|
SAP GUI |
SAP GUI is not supported. Selenium only supports web applications and it’s not possible to access elements in desktop applications outside the browser. |
Works across all technologies. SAP GUI is natively supported. |
|
Windows applications |
Windows applications outside browsers are not supported. |
Works across all technologies. All Windows technologies such as WinForms and WPF are natively supported. |
|
Citrix applications |
Citrix applications are not supported. |
Works across all technologies. Citrix and other virtualization technologies are supported with advanced text and image recognition capabilities. |
|
Mainframe applications |
Mainframe terminal applications are not supported. |
Works across all technologies. Mainframe terminal applications are supported with advanced text and image recognition capabilities. |
|
Java, IBM, and Oracle applications |
Java applications, including IBM and Oracle Forms are not supported. |
Works across all technologies. Java, IBM, and Oracle Forms applications are natively supported, including the older Java 1.6 standard. |
Test Execution and Integrations |
||
|
|
Selenium IDE |
Leapwork |
|
Scheduling |
No scheduler is included. Programmers can use a unit-test framework in combination with a custom-built scheduler or build pipeline. |
Scheduling and ad-hoc running is included out of the box. Flows can also be triggered from a build pipeline or other third-party system. |
|
Parallelization |
Can run in distributed, parallel environments, but requires programmers to build a custom automation framework (typically takes 6-9 months). |
Can run in distributed, parallel environments out of the box, including on BrowserStack, Sauce Labs and Selenium Grid as well as on Leapwork agents, where web, desktop and virtual applications can be automated together. |
|
Built-in REST API |
No REST API is included, but may be made available at a later stage. |
A full-fledged REST API is included out of the box, making it easy for DevOps to integrate with any third-party system. |
|
Built-in CI/CD plugins |
Any CI/CD platforms can be used by custom-built solutions by professional programmers. |
All popular CI/CD platforms can easily be integrated by DevOps using plugins or the REST API. |
The Complete Guide to Automating Tests with Selenium
Read the Leapwork guide to learn more about Selenium as a test automation solution and its benefits in testing. The guide also covers the skills, effort, and resources required to fully succeed with Selenium. Finally, it proposes a web automation solution you can use without having to worry about the technical details.
See the full extended comparison chart with the most popular test automation solution in our tool comparison.
