accelQ is developed on a design-first approach, where modularity and reusability are built into your test assets from the get go. Various use cases across accelQ, you will find this design-first philosophy as a central theme. Lack of maintainabilty of test cases has been the number one challenge in test automation and good design practice is essential for a sustainable automation.
Here are some best practices and tips you can follow, for further enhancing maintainability of test scripts and ensuring reliable test execution.
Project granularity and Taxonomy
A Project in accelQ typically represents one application-under-test, whose test assets are managed in one place. Most obvious mapping is that an accelQ Project corresponds to one software application that you are testing.
In some cases, an accelQ Project may also include multiple applications belonging to a portfolio, in case testing this portfolio most commonly requires navigating through multiple applications. If a user story for the portfolio cuts across multiple applications and thus involves testing these applications as a unit, you may design this suite of applications in one accelQ Project.
Project in accelQ should not be setup to correspond to a set of Requirements, Sprints or Releases within a given application-under-test. Nor should a Project be setup per User.
Setup Custom Fields to reflect organizational governance
In accelQ, you can setup custom fields for various entities. These custom fields help with test prioritization, governance and process management.
- Allow filtering the Universe based on combination of Custom Fields
- Setup Dynamic Test Suite based on matching Scenario filters with Custom Fields
- Setup Test Case filters when executing test suites
- General filter/update cycle on entities in bulk mode in the Navigator
- Breakdown the test case pass/fail based on Custom Field definition
You can setup different types of custom fields and assign these fields to one or more entities (Test Suite, Scenario, Test Case, Context, Action and Data Type).
Here are some examples:
|Custom Field||Values||Applied to|
|Priority||High / Medium / Low||Scenario and/or Test Case|
|Test Type||Regression / Sanity / Smoke||Test Case|
|Module||Flight / Car / Hotel||Scenario|
|App UI||Legacy / V2||Context|
|Criticality||High / Medium / Low||Action|
Custom field at Scenario level or test case level?
If you recall, a test case in accelQ represents an "instance" of a Scenario which comprises of a set of data values to drive the workflow designed in the Scenario. When you are designing custom fields, a question may arise if the field is applicable at Scenario level or test case level.
If your test design involves good amount of data driven possibilities with each scenario yielding multiple test cases, it may likely the custom field is more appropriate at test case level.
For example, consider the following custom field:
Test Type: Regression / Sanity / Smoke.
You may use this custom field to define a suite for Sanity testing. If your filtering granularity should be at the level of a test case, then apply this custom field at Test Case level.
Adapt a consistent naming convention for entities depending on your organizational standards. Unlike traditional automation platforms, accelQ provides greater flexibility in using non alpha-numeric characters including white space character, so that you can name your test entities more natural and meaningful. Don't feel the compulsion to abbreviate or use programming conventions such as upper camel, lower camel etc. Clarity and readability starts from the naming convention. Go ahead and provide meaningful names to your test entities such as Contexts, Actions and Scenarios.
Some examples here:
|Context||Purchase Confirmation Page||confirmationPage||Complete page name so there is no ambiguity. Use spaces for clarity.|
|Action||Verify Account Balances||Account Balances||Action is a "verb" and should represent some "doing"|
|Login to QBank||Enter User Name||"Enter user name" is just one of the statements as part of Login.|
|Create New Task||Creating new task||Action should indicate a "command verb"|
|Scenario||Update password and attempt login with old password||Update password||Be clear with the name to convey complete understanding of what the Scenario does.|
|Test Suite||Sanity Suite for Retail Banking||Sanity Suite 1||Identify the purpose of the suite and module etc|
Define appropriate status field values
Status field can be used as a great resource to track progress and readiness of entities. As a Project Administrator, think about your organization's governance and tracking needs and appropriately setup these status values. Make sure the team consistently tracks the status up to date for all entities.
Order the status list options in a logical sequence of progress indication. Specifically, the first Status list option is used by default, for all the newly created entities. For example, consider the list below (in that order):
- Work In Progress
- Under Review
Each Status list option has an attribute which indicates if an entity in that particular status can be considered ready for test execution. This readiness indicator attribute is used as one of the parameters when calculating the Test Suite readiness. Following a consistent approach here will provide great guidance on your Sprint or Release progress.
In the above list, if you believe only "Completed" entities can be considered as "Ready", indicate appropriately.
Alerts across the system
Alerts across various entities such as Actions, Contexts and Scenarios can be a great source of identifying incomplete items or those lacking common best practices. Pay attention to the alerts and try to address in a timely fashion. You can always filter the entities based on Alert flag from Navigator grids.
A Test Suite also allows you to filter Actions and Scenarios based on alerts.
Action with multiple destinations can avoid code duplication
Lot of times, you may have an Action resulting in different destination contexts based on some business rules. Using the multi-destination feature can avoid logic duplication by utilizing the same action under multiple conditions. You can find some examples here.
Avoid duplicating same logic in multiple actions
If you find a situation where a set of statements (or logic) is duplicating in multiple actions, think about extracting this logic into a separate action. These statements may not make sense from a Scenario or Business Keyword point of view, but you can still extract this as a separate Action for the purpose of reusability.
You can call this action from several other actions as needed. With an ability to share an action, you can make use of this common action across multiple contexts also. This promotes reusability and greatly enhances the maintainability of test logic.
Here is an example, where an Action, "Verify Transfer Amount and Confirm" is calling "Verify Footer Links" as a called-Action. Footer link verification may be useful in many other Actions and hence that logic is centralized in one Action.
Local actions can eliminate clutter and improve re-usability
Many times, you may define Actions to be used internally in other Actions (Local Actions). such "called" actions may not be of significance from the Scenario point of view. It is best to turn-off Scenario visibility of such actions. This avoids clutter when you are building a scenario. Remember, you want to provide as clean a list of keywords to the Scenario developer as possible. Small and incremental practices like these have a significant impact on the overall clarity and maintainability.
Synch point for a context is important
Setting up Synchronization point for a Context is very important. During test execution, it provides an automated way to ensure the page is fully loaded before acting on it.
At the same time, when the desired page is not loaded during test execution, you can avoid multiple wait times. Typically system waits at each statement looking for element relevant for that statement. This cumulative wait time can considerably delay test result, which is totally avoidable by setting up a sensible synchronization point at the context level.
Synchronization point provides an easy and implicit check point to abort the test case in case of application errors.
Use functional Wait logic in Actions – avoid hard coded wait statements
It is a good practice to insert a synchronization statement (wait-*) at the beginning of every major action to make sure necessary pre-conditions on the screen are available. Many times, automation execution reliability is impacted due to timing issues. A statement in your action logic might attempt to act on an element before it is fully rendered on screen. In accelQ, you can insert wait statements based on element availability, state of enabled/not, attribute value, css attribute value etc (functional waits). Here is a detailed article.
When you insert a wait statement, depending on the need, you have an ability to abort test case if desired state is not achieved.
As much as possible, avoid hard coded fixed-time wait statements. In some cases though, it may be particularly tricky to determine the readiness of an element. If you do need to insert a hard-coded wait statement, limit it to a maximum of 5 seconds (after all other avenues with functional waits are exhausted).
Design Test Suites with your execution requirements in mind
Plan your release or sprint level test executions through Test Suites, and not as individual Scenarios. With the Filter based and Requirements based suite design, you have an ability to design dynamic test collections. Few parameters that could make good filter criterion include Application Module, Sprint number, Build verification attributes such as Sanity, Regression etc.
You can also utilize Test Case filters to gain additional control on the level of granularity in suite design. For example, you could setup a suite with Scenario filter on Application Module, and then further division between Sanity, Smoke and Regression through Test Case filters. Remember, you can setup Test Case filters even for a Static Test Suite.
Here is a sample Custom Field design to accomplish above requirement.
Plan and organize your test executions just like test development
When you are in a release cycle, it is a common occurrence that automation executions are not particularly well planned or organized. Most of the time, tests are run in an ad hoc, incremental fashion and an automation team member takes the responsibility to collate and communicate the overall result.
In accelQ, test executions can be planned and organized equally well as the test development. In the release planning stage, determine which suites will be executed and on what schedule. Each cycle of execution should be well identified with a properly indicated "Purpose of Run".
In case there are reasons to re-execute part of the test suite in a given cycle (either due to build issues or other reasons), utilize the Re-run facility in the test report. When you re-run a test, all the instances (or attempts) of executions in a given cycle are collated together. You get a unified view to test report, which can be shared with the project team.
You will find "Re-run" button in the top right corner of a test report.
Make use of data driven testing and avoid Scenario duplication
A Scenario in accelQ represents a unique flow or use case in test application. Never write two Scenarios that are comprised of same sequence of Actions. All the data permutations within the same flow should be addressed by way parameterization and data driven testing. Do not duplicate or clone Scenario, unless you are creating a different flow/sequence of steps.
accelQ takes data driven testing to the next level, by allowing you to define parameters based on Data Types or Data Lists. When you find a particular Action parameter can only take a limited set of values, define appropriate Data Type or Data List for that parameter. You can automatically create permutations of test cases based on permitted values.
Context is not just limited to a page on your application
A Context is a logical entity that represents a particular state of you application. Although in majority of cases it may be equivalent to a page on the application, there are quite a few exceptions. You may be able to group multiple pages in one Context if all these pages are intimately related and cannot probably accomplish any consistent action independently. Group them into a single Context and have multiple Views associated with it. An action in this Context can freely make use of all the associated views.
On the same note, it is important to realize that minor state changes to a page do not warrant definition of new context. For example, when you hover on a doormat menu, you may be presented with additional fields you can interact with. This is just an additional View to the same context.
Proper design of Context boundaries allows better modularity and reusability. Here are a few articles to help with this decision.
Action sharing is a powerful feature that can avoid a lot of code duplication. Common navigation actions and business actions such as Logout are applicable in most pages in the application. Define these actions in one of the Contexts and share with rest. You will be managing element definition and other dependencies in a central place. Consider following two Scenarios.
Scenario 1: Login and Logout - Makes use of Logout functionality in Account Summary Page
Scenario 2: Update User Email - Makes use of Logout functionality from Profile Info Page
Perform Logout action can be defined in Account Summary Page and shared with the Profile Info Page.
Make use of Application Environments
accelQ supports the concept of Application Environments to mirror multiple deployment environments you may have in your product development process. As you promote the build between different environments, this feature allows you to seamlessly execute the same scripts without intervention.
Defining the list of Application Environments is the first step in this process. A Project Admin can review and setup the environments based on the specific project need.
Once defined, Application Environments allows you to abstract the data differences between different environments such as Prod, Test and Dev. Test execution in accelQ should be completely hands-free without requiring any environment specific updates when you run the tests. You can also conditionalize action logic based on the environment name.