Sikuli IDE

Sikuli IDE is a cross-platform application that enables you to control interfaces of any program running on your computer using screenshot images and the Python scripting language. Practically, it can capture a snapshot of a GUI element and automatically generate a script to perform certain actions.
The application is not difficult to use, but the explicit documentation is there to help you in case you get stuck along the way. Its interface is simple, comprising accessible and clear options, which makes it very intuitive.

A rich set of commands

The command list is available in the left pane, containing all the functions that you can insert into the script, which will have a captured image as the argument. You can find a certain element on screen, wait for a GUI component to appear on the screen or disappear, perform simple, double or right clicks, gain focus on the target window and then type or paste a text.

Use screenshots or images as function arguments

You start by taking screenshots of the target elements, so that Sikuli IDE can visually identify the elements to click on, drag and drop or hover the mouse cursor over. Alternatively, Sikuli IDE also enables you to insert locally stored images into the script and define screen regions.
The capturing mode can be activated using the dedicated button in the toolbar or the designated hot key. It freezes the screen and enables you to draw a rectangle around the desired area, inserting a small thumbnail in the script.
The code can be saved to file or to a HTML document, but not before you test it to see if it runs properly. You can launch the script and watch how it takes control over your mouse cursor and executes all the actions, in the defined order.

Use Python scripting to control interfaces

Sikuli IDE is a tool that can automate certain actions using Python scripts. The advantage is that it uses GUI elements as arguments for its functions, enabling you to control any application or desktop element.