For a while now, I've been involved in testing a complex media web application developed in Flex, and I got to the point where some help from a tool would have been much appreciated :). I have to admit that this was my first RIA project, so I decided to dig a little bit into it, especially regarding the options one would have in automating some of the functional tests.
Scope
I don't want to argue about the ROI of an automated GUI test suite, or about the fact that maybe it would be much more useful to automate some other types of testing like unit and performance (there are specific tools for each of these).
All the info from this post can be found on the internet, but it takes a lot of time and trouble to gather and analyze it. This post will provide a list of tools I found that support Flex Automation, hoping it will help you if you're thinking about functional testing automation for Flex based applications.
The Flash in the blackbox
The main issue with RIA Flex applications is that the Flash player is basically a runtime, which makes its objects invisible to its container (e.g. browser's html or javascript code), unless explicitly exposed from within.
I was able to identify three ways to add automation support for Flex applications:
Scope
I don't want to argue about the ROI of an automated GUI test suite, or about the fact that maybe it would be much more useful to automate some other types of testing like unit and performance (there are specific tools for each of these).
All the info from this post can be found on the internet, but it takes a lot of time and trouble to gather and analyze it. This post will provide a list of tools I found that support Flex Automation, hoping it will help you if you're thinking about functional testing automation for Flex based applications.
The Flash in the blackbox
The main issue with RIA Flex applications is that the Flash player is basically a runtime, which makes its objects invisible to its container (e.g. browser's html or javascript code), unless explicitly exposed from within.
I was able to identify three ways to add automation support for Flex applications:- using the Flex Automation Framework (+ExternalInterface). Adobe has developed this framework to provide a way for developing ActionScript automation-agents that facilitate the communication between the automation tools and the Flex objects. This is possible using the ExternalInterface API. I couldn't find a way of using the Automation Framework without the ExternalInterface, although it is not specified as the only option for establishing the communication. Even though the Flex SDK is open-source, the Automation Framework is not: the trial version can be used for scripts with up to 30 actions. One would have to pay 499€ (VAT not included) for the FlexBuilder Pro license in order to create larger scripts.
- by developing your own framework in ActionScript and make it visible through the ExternalInterface - more info about the API can be found here.
- IAccessible interface and MSAA. Not all the objects support the IAccessible interface by default; here you can find more info on how it can be added.
The Tools
Below you can find a table with the tools that could be useful. I say this because I found some tools that claim to support Flex Automation, but they are just basic macros (mouse click at a certain coordinates on the screen), i.e. iMacros from iOpus.
We plan to take a closer look at some of these tools, especially at the open-source ones but also at the ones that have a decent price. I am pretty sure that companies like IBM, Borland or HP have their own research and analysis on flex automation already available. Plus, we are not a big corporation, so we don't plan to acquire any of them in the near future :).
Below you can find a table with the tools that could be useful. I say this because I found some tools that claim to support Flex Automation, but they are just basic macros (mouse click at a certain coordinates on the screen), i.e. iMacros from iOpus.
We plan to take a closer look at some of these tools, especially at the open-source ones but also at the ones that have a decent price. I am pretty sure that companies like IBM, Borland or HP have their own research and analysis on flex automation already available. Plus, we are not a big corporation, so we don't plan to acquire any of them in the near future :).
Here you can find some useful info on how to setup your environment and get started with automating your applications.
Stay tuned, we'll post some new info soon.
Stay tuned, we'll post some new info soon.
19 comments:
Interesting post. Thank you!
It's strange that you did not include FlexMonkey, which, imho, has gone a bit further than other open-source solutions you listed.
Here is the google code page:
http://code.google.com/p/flexmonkey/
You are right, I don't know how I could have missed it, as it seems to be the only one fully based on ActionScript.
I updated the table, and I will look into it more closely.
Thanks for pointing it out!
I've found you guys on kiva. Congrats on your initiative and on growing the Ro team.
I have to tell you though, I did not expect something this technical.. I'll be back as soon as I have more time.
I'm very happy to see a business that didn't buy into the Wordpress cult :)
How to test a web based application with an embedded Flex app using Flex monkey?
I'm thinking that FunFX - Watir based - or Flash Selenium/SeleniumFlex - Seleneium based - might be an easier solution as both Watir and Selenium are widely used for testing Web Apps.
If you really want to use FlexMonkey, then you need to make your test methods callable from the web app through ExternalInterface.
Hi
Please give me a solution for automation testing of web application (which is developed in java and flex)
Before I can provide some suggestions, I need a few details on what you are trying to automate.
1. What exactly do you want to automate (what do you expect to get at the end of the testing)?
2. Is it not possible to get the same results by running manual tests instead?
3. How much of your web app is developed in java and how much in flex? Is it a java based app with an integrated flex app?
4. Is your application developed with test automation in mind?
5. Wouldn't it be easier to test the flex app separately and after the integration to do a manually system test?
These are just a few questions that came into my mind.
My point is, in order to have a solution for your problem you need to gather/provide more info related to it.
Thanks,
Alex
Alex could you please include the supported compilers as well (FLASH PRO / FLASH BUILDER / FLEX SDK ).
Thanks and regards,
John
Hi John,
I apologize for the late reply - it was a busy time lately.
You need the Flash Builder Pro license to use the automation framework for unlimited number of actions. If you don't have the license, you'll have to limit your scripts to 30 actions/script.
Alex
Hello Alex,
is it possible to do test automation of a flash application build without the use of Flex Builder? The application I want to test is build on base of ac2 (no Flex builder) and don’t us UIComponents. The flash files us MovieClips for the buttons. There is also no MXML file in use. We use QTP but I im not able to find a way to recognize Flash elements.
Thomas
Hi Thomas,
Here are the first things that came into my mind:
1. Did you include the test automation libraries (automation.swc, automation_agent.swc, ...) when you built?
2. If not, are you using a runtime loader that was built with the automation libraries?
For more info on the setup you need to do, please see this post: http://blog.altom.ro/2009/03/setting-up-your-environment-for.html
Alex
Hi Alex,
Can you please help me out. Can you tell me that Does FlexMonkey can automate Flex Application which has Customized flex objects?
Hi Ashish,
If you extended an already instrumented component (such as combo-box or button) it should work without you needing to do anything.
If your component inherits from UIComponent, then you need to instrument that class for testing capabilities - you can find more info about this here: http://livedocs.adobe.com/flex/3/html/help.html?content=functest_components2_02.html - Instrumenting Custom Components.
Either way, my advice is to use unique ids for your components, otherwise FlexMonkey will identify some unique id for each component, but I find them hard to use.
Thanks,
Alex
Thanks alot ALEX.. This information is very useful for me.
Hi Ashish,
I'm really happy that I could help.
Alex
Hi Alex,
I'm trying ( struggling frankly ) to use flash-selenium for automating flash applications.
Now my doubt are as below
1. I have access to all .as files
2. I'm planning to write custom class on top of these and make my function ExternalInterface for javascript calls.
Can I use this swf file locally and call these functions?
I'm trying it but it's not working, can you please share your email id so that I can share my sample code with you
thanks in advance.
regards,
Vikram
You can find additional alternative, see the following:
http://auto-simple.blogspot.com/2010/02/test-automation-for-flex-application.html
Hello Alex,
Thank you for your post.I am working on flex/flash testing project and I need to have an automation framework. I am trying to have most reliable tool which can help me even if application keeps changing (updates). But here are few problem or say conditions which I have to follow,
1)My application is web-based(GUI) and I do not have control on its build. So can not add any libs.
2)Product is already halfway under development and now I can not change it to make it automation friendly.
if you can help me out, it will be really great. Thx a lot again.
Hi Alex--
You left out a product. eggPlant from TestPlant (www.testplant.com) works really well on Flash/Flex applications. It uses image recognition technology so it doesn't even care that it's a flash or flex app, or whether it's standalone or browser-based. I've been a user of the product for many years, and joined the company in a support role last year. I highly recommend you check it out. It works differently than any other tool out there.
Thanks!
Allen
Post a Comment