With loads of legacy Flash/Flex code where test automation support is limited or in most cases even non-existing there is only so much that can be done in forms of automation. I cannot understand why I did not pick up Sikuli earlier but I should have, that is all I am saying. After two days of playing around and demoing QA engineers around me are so eager to get going.
The concept of image recognition and test case maintenance does not sound too tempting but for the case with legacy flash implementations where the graphics will surely not change it would be unfair not to try it out. So we did but it was not out of the box.
The simplest of test cases
In short, unless you want to read it from Project Sikuli. Sikuli can match a predefined image to a section of the screen that is VERY similar to predefined image, ‘best fit’. So for example if you take a small screenshot of an object in you Flash implementation and save it as an image Sikuli can then be used to interact with this object, clicking on it, dragging it, checking if it can find it on the page asf..
Sikuli can rather easily be tested out, for example against a common Flash application like the one in the clip below. To get a test cases running just crop out the correct pictures from the application under test.
The pictures below is found in the application under test without any problems for Sikuli.
Playing roulette using Sikuli.
The Junit test flow looks like this.
// Java
//
// Example Sikuli Test flow, no verifications in place it only reflects what is
// visible in the youtube clip above.
//
@Test
public void playRoulette() throws Exception {
SikuliDemo sikuliDemo = new SikuliDemo();
sikuliDemo.clickButton("bet5.PNG");
Thread.sleep(10000);
sikuliDemo.clickButton("betonred.PNG");
sikuliDemo.clickButton("bet5.PNG");
sikuliDemo.clickButton("betonred.PNG");
Thread.sleep(10000);
sikuliDemo.clickButton("cleartable.PNG");
Thread.sleep(10000);
sikuliDemo.clickButton("betonodd.PNG");
sikuliDemo.clickButton("bet5.PNG");
sikuliDemo.clickButton("betonodd.PNG");
Thread.sleep(10000);
sikuliDemo.clickButton("bet25.PNG");
Thread.sleep(10000);
sikuliDemo.clickButton("beton13.PNG");
Thread.sleep(10000);
sikuliDemo.clickButton("spin.PNG");
Thread.sleep(300000);
}
Integrating into your Java test framework
So how to, using Java? In fact code wise, very little code is needed to get the basic functionality in place, click and verify. The code sample below simply looks for an image snapshotted from the application under test anywhere on the screen and clicks it as well as checking for the visibility of another. Just make sure that sikuli-script.jar is in the classpath.
// Java
import org.sikuli.script.Screen;
public class SikuliDemo {
public boolean clickButton(String gameName, String imageName) {
// Window is already opened by WebDriver
Screen s = new Screen();
try {
s.exists(imageName, FLASH_TIMEOUT);
s.click(imageName, FLASH_TIMEOUT);
} catch (FindFailed ff) {
return false;
}
return true;
}
public boolean verifyImage(String gameName, String imageName) {
// Window is already opened by WebDriver
Screen s = new Screen();
try {
s.wait(imageName, FLASH_TIMEOUT);
} catch (FindFailed ff) {
return false;
}
return true;
}
}
… not as easy as expected, the following issues are some that surfaced.
Running on a 64bit Windows system
Be thorough when configuring your system or it is very easy to get error messages relating to Win32Util.dll or VisionProxy.dll. I basically had to uninstall all previously installed Java versions to get it working.
A Java SDK/JDK 32bit installation is needed, these end with ‘i586’ on the Oracle download site. Then make sure the following environment variables are set accordingly on the running system.
- SIKULI_HOME – Sikuli installation folder
- JAVA_HOME – To the 32bit installation
- PATH – Add %SIKULI_HOME%\libs;JAVA_HOME\bin
Running from Jenkins
The next obstacle, Sikuli needs access to the desktop to work. The most common way to run Jenkins is a Windows web service, yes it can be configured to get access to the desktop but no matter how we could not get it working properly. The solution that worked out in the end was to run Jenkins standalone from a command window using;
$ java -jar jenkins.war
By doing that Jenkins had access to the desktop as well as making it possible to watch the tests execute when logged in to the build agent.
Firefox in full screen
Some tests still had problems running smoothly on some machines due to screen resolutions and visibility. Since the Flash was embedded into a web page setting Firefox to expand to full screen was a fix that worked out quite smoothly. Using the WebDriver and to get the FirefoxDriver simulate an F11 key press was done with the single line of code below.
// Java
((FirefoxDriver)driver).getKeyboard().pressKey(Keys.F11);
Downloads
Technical testing: Lessons learnt // Walkthroughs // Hints --- for the technical tester 