Availability: Release 6.0
When running an Automation Scenario or Suite, ACCELQ provides the ability to set up Auto Re-run configuration. This capability is part of Smart Reconcile options in ACCELQ.
At times, a long-running automation test suite may experience flakiness either due to environmental issues, data sanity, application logic with concurrent transactions, etc. Such failures may usually get auto-corrected on a re-run of the test. When you kick off a test, ACCELQ provides an ability to set up auto re-run configuration to address such issues.
Setting up Auto-Rerun
In the Run modal, click on the "Smart Reconcile" section.
The following configuration options are available:
- Maximum Re-run attempts: Indicates the maximum number of re-run attempts that should be made. Allows a value up to 9.
- Invoke Auto Rerun if test case failure exceeds: Allows you to control if Auto Re-run should be invoked only conditionally when the failure count exceeds a certain limit. Input this number as a percentage of the test case failure of the original run. If you need the auto re-run to kick off unconditionally, regardless of the failure count, input this value as 0.
- Delay between each re-run attempt: Amount of time to wait before kicking off each re-run job. Comes in handy in situations where you expect the application environment to stabilize after a delay in execution.
- Stop further re-runs if the last re-run has no improvement: This flag indicates if further re-run attempts should be conditional based on the preceding re-runs showing any improvement. You may save unnecessary re-run attempts by aborting the attempts in case no improvement is shown in the prior re-run.
Auto Re-run kicks off as a batch, consolidating all the failed tests after a given iteration is done. In other words, if a job has 10 test cases, the system runs all the test cases one by one, while keeping track of failures. Once the original job is done, the system invokes a fresh (child) job with all the failed test cases in one batch.
Multiple re-run iterations create individual reports for each iteration. ACCELQ consolidates all these re-run reports into a unified view with the latest instance of each test case execution pulled in.
By default, when you open the report for the job, you get a view of the consolidated report. You may also access individual re-run jobs reports if needed. Refer to the Consolidated Report section in this article for more information.
Auto Re-run is applicable in CI/CD executions as well. Simply configure the feature while creating the CI/CD template job which is kicked off from the pipeline.