The tutorial below walks through an example of how to use the Clipboard in Windows as a tool to handle challenging text recognition scenarios in virtual desktop automation.
Learn how to:
The ability to read text and numbers from the screen is a cornerstone in virtual desktop automation. This is enabled by technology called OCR (optical character recognition) – or simply “text recognition”.
OCR is based on recognizing patterns from the screen, and although it is a robust and proven technology, there are certain situations where OCR has its limitations. Imagine a text in a field, where the text color is black and the background of the field is grey. This is not a problem for the human eye, but it will have an effect on the OCR engine's ability to recognize the individual letters.
Please note that the tutorial video involves an older version of the LEAPWORK Automation Platform (formerly called LEAPTEST), but the principles shown are still valid.
The example in the video shows how to use a combination of the position of a label and a Drag Mouse building block to select a text in an application. The text is located in a non-editable field with a dark grey background, which is a known challenge for any OCR engine.
The idea is to select the text in the application, instruct the operating system – in this case Windows – to copy the selected text to the Clipboard, and then use the Get Clipboard building block to read the text copied to the Clipboard.
Using the copy-paste patterns is a "poor man’s solution" for selecting “objects” in a virtual desktop application. Instead of reading the text directly from the screen using OCR, the pattern instructs Windows to read the value from an object (the selected text), and then the text is transferred using the Clipboard.
Go back to tutorials for virtual desktop automation.
.svg "Leapwork")
