Mainframe Green Screen Testing:
The Ultimate Guide

Mainframe may be a term you’re not quite familiar with. And that’s understandable because there are many terms and concepts associated with mainframes. We’ll briefly dig into those below:

Related in-depth reading: What is Mainframe Automation?

• Mainframes (e.g. IBM Z Series). A high-end server from IBM that runs centralized applications and/or stores large quantities of data. They’re commonly used by 30+ y/o financial organizations.
• Mid-range servers (e.g. IBM AS/400 or iSeries). A cheaper, less powerful alternative to mainframes. The challenges and impact of testing are just as relevant and the same automation approaches apply.

Related reading: What’s the Difference Between AS400 and Mainframe?

• Mainframe emulation software (e.g. Hercules). A lower price point than a mid-sized server. It runs on a local computer running Linux, Windows, or macOS and imitates a mainframe, however, data is stored by a virtual driver.
• Green screen/terminal emulation software (provided by e.g. IBM and Rumba). These days, green screen (or “legacy”) refers to terminal emulators running on a personal computer. Put simply, green screen = terminal emulation. Terminal emulator software lets people communicate with a centralized computer (e.g. a mainframe).

How are all these terms connected then? A green screen (otherwise known as terminal emulator software) will run on a PC. It connects to a mainframe emulator, a mainframe, or a mid-sized server.

While reliable, scalable, and great at supporting masses of data and transactions, mainframes are slow to change. They have changed very little over the last 20-30 years, and the processes for developing and testing them have also remained somewhat unchanged. This prevents businesses from keeping pace with competitors, and it prevents agility.

In the next section, we cover what businesses are doing to overcome the transition from traditional to modern approaches to testing.

From manual to automated tests: how to overcome common barriers

The pool of available developer resources who can develop mainframes is draining. But the workload and expectations for quality at speed are growing. Manual testing can’t keep up and it forces testers and developers to miss important parts of testing which sacrifice quality. 

Related reading: Mainframe Testing & Why Now is the Time to Automate

For banking, financial services, and insurance (BFSI) industries particularly, the transformation towards becoming digital-ready is not an easy one. The processes for testing mainframes are just as old as the technology itself, meaning innovating can prove to be difficult. 

To meet the demands of the digital world, mainframe testing and development have to be modernized. One way to achieve the efficiency needed is to move away from manual testing and adopt newer practices and tools, like shift-left testing and test automation.

Shift left testing means testing is done earlier in the software development lifecycle, and more often. The point is to catch bugs before they make it to production so that they are less expensive to fix.  

Why are businesses shifting left testing in mainframe?

Shift left testing has grown in popularity within mainframe application testing. This is partly due to the need to find a better and more efficient way to test mainframes, because:

1. There are few mainframe experts. The number of people with expertise in mainframe is rapidly declining. Few people are continuing to build their skills. This makes building automation for testing much harder. 
2. Delivering more, and faster. Businesses are being pushed to deliver more, faster. DevOps and Agile practices are helping them achieve that speed, but these practices don’t mesh well with traditional approaches to mainframe testing. 

Why test automation helps businesses shift left

Test automation is the key to successful shift-left testing. It helps businesses test faster, more often, and with higher accuracy.

Related reading: Shift Left Testing for Mainframe: Why and How

Faster testing

Done manually, testing takes time and it is repetitive. 

Done with automation, tests take some time to set up, but once they are in place, testers are freed from tedious manual testing. They run 24/7 and can be set to work simultaneously so you can get more test results, faster. 

A codeless automation tool cuts down the initial setup even more to reduce the time spent testing. What’s more, anyone has the ability to set up testing as you don’t need to script to build an automated test. By removing the dependency on code, teams can complete their release cycles quicker. 

Better quality

When manually testing, it’s impossible to get good enough coverage to be sure that your system is working as it should. 

With code-based automation, the time it takes to build a functioning script and the maintenance required can often outweigh the benefit of using automation altogether. It can quickly become a time sink. 

Codeless automation makes it possible for testers and business people to set up tests, run them, and maintain them with ease. Developers have more time to spend on mainframe development because they’re not building and maintaining test automation. Testers have more time to spend on exploratory testing and test design. As a result, quality is improved. 

Reduced costs

Bugs become more expensive the later they are discovered in the release pipeline. Shift left lets you test even earlier, meaning you’ll find bugs before they hit production, keeping costs low.  

Coupled with codeless automation, defects are found earlier and faster, and developers are able to avoid building on top of broken code. This results in a transition away from outdated approaches to development, and towards more efficient, cost-effective releases with higher quality and lower risk.

To many, however, testing earlier and test automation don’t go hand in hand.

The test automation paradox

The test automation paradox is the inability to set up automation until test cases are complete. It takes a long time, and it depends on resources that most teams can’t justify. 

That doesn’t mean that test automation can’t bring you faster testing, a better quality mainframe system, and reduced costs. It can, but it requires the right culture, processes (like agile and DevOps), and tools. 

However, without automating mainframe testing:

• you’re more likely to find defects later in development or production
• release cycles will be inefficient and reliant on scarce developer resources
• surrounding applications may experience unstable performance, opening up security vulnerabilities.

In the following sections, we'll explain what options are available for testing mainframes with automation. 

To be able to automate mainframe green screens you need an automation framework or tool which either relies on Optical Character Recognition (OCR), code, or built-in support for terminal emulators. 

An automation framework is commonly code-based and requires coding language to build. This makes them more resource-dependent than OCR or built-in terminal emulator support. 

We’ll describe these two approaches in more detail below: 

• Optical Character Recognition (OCR). OCR identifies elements on a screen and validates that the user interface (UI) and elements on a green screen are appearing as they should. It comes in handy when code is difficult to access. But it is time-consuming to create and maintain automation with this approach. For example, if an element moves, the OCR engine might not find it and the test will fail, even if the process being tested is working. 
• Built-in support for terminal emulators. A more stable approach is to use a tool that offers built-in support for the terminal used by your company, whether it’s Rumba+, ICM, or Hercules. 

Can Selenium be used to test mainframes?

One common query is whether Selenium can be used to test mainframes. 

Related reading: Can You Use Selenium to Test Mainframes?

While it is possibly the most well-known open-source testing tool, it cannot be used to automate mainframes. This is because Selenium can only automate web-based applications, not mainframes.

However, there are alternatives that give you the same capabilities, while also working across web, mainframes, and desktops. 

While there are lots of ways you can automate mainframes, you also need to consider what you should and can automate. From unit testing to regression testing, we detail these below. 

Related reading: How to Do Test Automation in Mainframes

• Unit tests. Testing secluded components of a larger system. This could be testing COBOL programs and single paragraphs of code. 
• Integration tests. Testing integrations between applications outside, or within the mainframe. They’re a key step, but they are often cast aside because of time needed to conduct them, especially if this process is done manually.
• System tests. Testing the full system to check whether it complies with pre-set requirements. You will be testing the mainframe and all applications connected to it, for example, the database, the user interface, and transactions. 
• Acceptance testing. Considered the last step before delivery, acceptance testing will determine the readiness of the system and whether it functions for end-users. Because it is sometimes performed outside the development and testing team, it’s often done manually by product managers or other relevant business users. This makes it a perfect candidate for codeless automation, as anyone, regardless of technical experience, can build automated tests. 

The transition to becoming digital-ready can be challenging, especially when the same processes for development and testing have been relied on for decades.

But there are steps that can be taken to make sure that the transition to modern mainframe testing goes smoothly. 

Related reading: Test Automation Tools for Mainframe Applications & What to Look 

Part of this transition encompasses the type of automation tool you use for testing. This has a large influence over how fast the transition will be, the quality of development going forward, and how the time and skills of your team are distributed to achieve an agile mainframe development model. 

The two most important capabilities to look for are:

• Cross-technology capabilities. Mainframes are not used as stand-alone systems. They are often at the core of an IT infrastructure, and they will connect and integrate with technologies and platforms like SAP and Salesforce. In combination, they all work together to deliver value for the enterprise. From a testing perspective, the automation tool should be able to test end-to-end across integrations and technologies. This gives a better overview of when and where bugs are appearing. 

• Codeless automation. With no-code automation, there are a number of benefits. Technical and non-technical, mainframe developers and non-mainframe developers can build and update automation. Maintenance becomes much lower, and tests can be built faster. This eases the burden on scarce mainframe developer resources, making it possible for teams to scale automation on a level not possible with a code-based tool. 

Leapwork: A no-code mainframe test automation tool

With the right processes and people, mainframe development and testing can produce better results. The development timeline can be shorted, quality will improve, and critical errors will be discovered before they make it to production.

Leapwork's codeless automation platforms help businesses automate with ease, and faster. With an intuitive visual interface, you can automate across technologies, whether it's on green screen, desktop, or web. 

You can learn more about codeless test automation with Leapwork in our on-demand webinar: overcoming the challenges of automating green screen testing.