Overview of Test Report

Test Report in ACCELQ

ACCELQ Reports provide a functionally rich interface to test execution results. You can view the results in real-time as the tests are executing and the reports are auto-refreshed while you review. Test Reports provide a unified view to execution results regardless the test was run on a local lab, or an external infrastructure like Browser Stack, Sauce Labs, etc.

Test Reports in ACCELQ are functional as well as actionable. The following sections provide a brief overview of each of these features. In addition, it is a great tool for troubleshooting, with features such as contextual error help and slideshow display, etc.

Note on state transitions during test execution

During the life of test execution, various state transitions occur and are displayed in the top summary header. It is important to understand these transitions so that you can track progress appropriately.

When the test is just kicked off, a test goes into a Scheduled state until it is picked up by an agent to run. The following flow diagram captures this state transition.

Test Run information

Test Report displays the summary of run information at the top of the page. In addition, you can get to detailed information on the test run, along with the log files, etc. by clicking on the info icon in the top toolbar.

Local Agent status during test execution

While the test is executing, you can view the local agent's current status in terms of activity and CPU/memory resource utilization. You can view this information by clicking on the little desktop icon displayed beside the test purpose at the top.

Test Run Summary

Test Run summary is displayed at the top of the screen with the current state, purpose of run and execution time, etc.

You can also view the run configuration information in the right-hand side section right below the top summary.

Perspectives

Perspectives provide an ability to filter test reports based on business parameters. You can customize the list of business parameters and quickly attribute the pass/fail information to decision-making parameters. Helps with quick analysis and ultimately a go/no-go decision. More details here.

Test case listing and filtering

Test cases are listed in the left nav of the detailed report. Each test case has a pass/fail status indicator. This list is dynamically updated as the execution makes progress.

You can filter this listing in multiple ways:

1. By clicking on the overall pass/fail count in the test summary at the top.
2. By clicking on one of the bars in the Perspectives dashlet, you quickly filter the corresponding test cases.
3. By entering a search text at the top of the test case listing. 

Detailed test case report

A detailed test case report is displayed by clicking on one of the test cases in the left-nav listing. Entire test case execution information is rendered in this part of the screen. Execution time is displayed on the top along with the Scenario name from which this test case was generated. You can also navigate to the Scenario by clicking it here.

• Information is grouped on Actions executed during the run with a pass/fail indicator against each action. Execution time is also displayed against each action along with the Context to which the action belongs. You can click the action name and context name to open them in separate tabs.
• Every statement has an "info" icon to provide further information on the command used and the arguments supplied.
• If there is element interaction in a given statement, it is displayed with a hyperlink for further details.
• The screenshot icon against each statement shows the corresponding screenshot.

Viewing Slideshow of test execution

You can click on the slideshow icon in the test case detail section for a step-by-step image snapshot list from the test execution. Depending on the Screenshot option selected during a test run, application screenshots are available at different step granularity. When you click on the camera icon beside a statement in report detail, you open the slideshow with the current screenshot navigated by default.

Contextual help

Within the test case details section, when there is an error message displayed, you can find contextual help for troubleshooting. If there is contextual help available for a given condition, a glowing bulb icon is displayed by the statement. Click on it to find useful information on what the error indicates and some helpful tips to avoid the failure or to further troubleshoot.

Bug integration

When there is a failure detected in the report, and if it is required to open a defect, you can do so from within the report. Click on the bug icon in the test case where the defect is detected. It opens up a form with information pre-filled with steps to reproduce and error screenshots. It creates a defect in your target integrated systems such as Jira or TFS. You can view currently associated defects by clicking on the bug icon again or the link provided at the top of the report.

Re-Run test cases

This is a very powerful capability to re-run tests in case of failures. If a part of the test cases needs to be re-executed for any reason (such as application environment issue etc.), you can do so from the test report and all instances of execution reports are consolidated into the main execution report. This facilitates easy sharing and analysis of reports which is targeted for one execution cycle. More information about test re-run here.

Review report in verbose mode

By default, some of the non-essential information is hidden from display in the test report. In case you are reviewing the report for troubleshooting and if all finer details are required, click on the verbose icon in the test case detail header as shown below. This opens up all the hidden statements.

Aborting test execution

When a test run is in in-progress status and if is required to be stopped, you can abort the test from the report. You can accomplish this by clicking on the action menu in the top right corner of the test report.

Export and Email report

Report link along with a summary of test case status can be emailed to a specified list of recipients. You can also export the report in pdf and excel formats for further sharing. All these options are available from the top-right action toolbar.