Analyzing Manual Test Report

Availability: Version 4.0

Test report provides detailed information on a test run along with tools to analyze the result for application quality. Both an individual Scenario run and a Suite run produce the same type of report for analysis.

Summary information

In the top section of the report, you will find the total number of test cases along with breakdown of test cases in various status such as pass, fail, blocked etc. Clicking on a particular status count, filters the test case listing in the detailed section.

Perspectives

Perspectives help you drill-down test results for further analysis from business perspective. You can define custom criterion by setting up appropriate taxonomy on Scenarios and Test cases. For example, when you see a set of pass/fail test cases, you might want to know the breakdown of such failures based on Requirement ID or Priority or Line of Business.

When applied, you can filter the report based on a Perspective, across hundreds of tests cases in the report. This helps in quickly analyzing pass/fail tests with respect to a business parameters and make an informed Go/No-Go decision. 

Perspectives are enabled in the report by using Custom Fields applied at Scenario or Test Case level. You can define custom fields in the project configuration and assign to Scenario and Test Case modules. For each Custom Field (Perspective), you can set multiple dimensions through a list of values. For example, you can set "High", "Medium" and "Low" dimensions for a perspective called "Priority". Click here for details. All the custom fields of type: List, Multi-select or User List are picked up for Perspectives.

When you are navigating in a test report, you can click on any of the segments in Perspectives graph and the test case list is filtered based on that segment. 

Test Case Detail

When you click on a particular test case in the report, detailed steps are displayed in the right side panel. It provides complete step by step status as well as any annotations that the user may have provided during test executions. If any screenshots were attached, you can access them at the step level.

From the top header area in test case detail, you can view a slide show of all the screenshots captured during the run. You can also open the list of defects that were created during the test run. If required, a new defect can also be associated from the test report.

Resuming test run

If the test run is still in-progress, you can resume the run directly from the test report rather than having to go back to respective Scenario or Test Suite. When you resume the run, you can make further updates to test case status which will be reflected in the report.

Exporting and Emailing results

From the test report, click on the ellipsis icon in the top right corner to view options for exporting report to various formats. You can export the report to PDF format, or an excel sheet. You can also email the report to external users.