Test Automation for Agile Environments

Amit NimjeMay 30, 2016

By design, development projects using the Agile methodology move quickly toward frequent, new product releases. In such environments, timely quality assurance (QA) feedback is a must. To be useful, the test results must also be consistent and reliable. This combination of requirements – speed, quality, and reliability – highlights the usefulness of automating testing in Agile environments.

 

Historically, QA during Agile-driven projects faced several challenges. One challenge is the sheer number of components that must be tested on short timelines. In the case of two or three simple dashboards, testing can be performed manually. But as project components multiply and complexity increases, short timelines make manual testing unfeasible, especially for features such as customized filter panels, tool tips, and calendar filters.

 

Another challenge is achieving 100% functional and data testing for each release. Aggressive timelines make frequent, often repetitive and redundant scenario testing difficult to complete on time.

 

Multilingual products present unique challenges to Agile teams. Suppose a dashboard supports nine different languages. It can be difficult to test nine versions of the same dashboard, each in a different language, for every new release. In these situations, testers usually focus on new features and the user interface (UI). Testing the language and data in each dashboard’s reports usually falls by the wayside.

 

To overcome these challenges, we developed a test automation utility using Selenium. We also employed Perl scripting to integrate the utility with database and web applications. Fig. 1 illustrates the framework for the utility.

 

test_auto

Fig. 1 Framework for Automated Testing Utility

 

As illustrated, the framework uses four data input sheets (i.e., the gray boxes inside the blue box), which define the testing flow for an application. The top layers contain data that are input into a Microsoft Excel spreadsheet. The spreadsheet contains four tabs, one each for user credentials, language selector, testing steps, and multilingual correlation testing (e.g., across print and PDF versions of text).

 

Once the input sheets are ready, the driver script is triggered by a batch file, either manually or scheduled, which in turn calls the required functions from the function library. The function library takes the inputs from the spreadsheet and returns output defined by the driver script. Finally, the driver script generates the output folder for individual execution with date and timestamp, which consists of “Pass or Fail” status for each of the four data inputs. The output folder also includes screenshots.

 

This utility supports different areas of QA, such as:

  • Data testing – Testers must verify data in the application (e.g., dashboards, grid reports, etc.) against source databases. Random sampling is often used for large data sets, but the utility can test a complete data set just as efficiently as humans can manually test random samples.
  • Functional testing – Client requirements are sometimes vague, and at times not supported by an out-of-the-box feature. So customization is common in meeting client requirements. But customization increases the functional complexity of the application. The utility performs complex and repetitive functional testing tasks.
  • Multilingual testing – Language fluency is prerequisite for manually testing dashboards in different languages. The utility eliminates this requirement by automating tests in all languages.
  • Correlation testing – Along with web-based dashboards, print and PDF versions of the dashboards require testing. The QA team can use the utility to verify textual consistency across web, print, and PDF versions of dashboards.
  • UI testing – The utility performs partial UI testing on dashboards, providing screenshots at any point during testing for QA and test documentation.

The utility provides numerous advantages, including:

  • Zero cost – Selenium is open source and can be downloaded for free at the Selenium
  • Flexible use – The utility is environment independent and can be run for applications other than QA, using the same input spreadsheets.
  • Time savings – The utility replaces the need for much manual testing, including redundant scenario tests on different languages and across data sets. The script can be scheduled to run overnight.
  • Scalability – The global function library supports many use cases on multiple types of projects. Most adjustments for different uses require minor modifications to the input spreadsheets.

If you work in an Agile environment and are searching for better ways to implement effective QA, contact us. We will be happy to demo this utility or to discuss developing another solution for your requirements.