Running a collection of Scenarios as an Ad-hoc Suite

An Ad-hoc Suite is a powerful feature that allows you to create a temporary, on-the-fly collection of Scenarios for immediate execution, directly from the Run modal. Think of it as a disposable test suite designed for specific, in-the-moment testing needs, without the requirement of creating and managing a formal Test Suite asset in your project.

This provides tremendous flexibility for your day-to-day testing activities, allowing you to dynamically group and run tests based on your immediate requirements.

When to Use an Ad-hoc Suite

While formal Test Suites are essential for structured regression, release validation, and historical analysis, Ad-hoc Suites are perfect for temporary or exploratory situations.

Consider using an Ad-hoc Suite for:

• Targeted Debugging: You need to quickly run a specific Scenario that failed in a larger run, along with a couple of related Scenarios, to verify a fix.
• Quick Sanity Checks: Under version control, before committing your changes, you want to run a small, cross-functional set of critical Scenarios to ensure no major functionality is broken.
• Exploratory Testing: You want to validate a new feature by running all Scenarios tagged with a specific custom field (e.g., Module: 'New-Payment-Gateway').
• One-off Requests: A project manager asks you to run a specific combination of Scenarios to generate a report for a demo, a task you're unlikely to repeat.

A Note on Best Practices: An Ad-hoc Suite is not a replacement for a formal Test Suite. For any execution that needs to be repeated, tracked over time, or included in your project's analytics and reporting, it is always recommended to create a saved Test Suite asset.

How to Configure and Run an Ad-hoc Suite

Building your Ad-hoc Suite is an intuitive process. Once you select this option in the Run modal, you can define your temporary suite either by manually picking Scenarios or by creating dynamic filters.

1. From the top toolbar, click on "Run" button and select Automation Run.
2. In the Run modal, find the dropdown labeled "What do you want to run?" and select Ad-hoc Suite.
3. Here, you have two methods to build your suite: By Name or By Filter. Select your preferred method using the toggle at the top of the window.

Selecting Scenarios by Name

This approach is ideal when you know the exact Scenarios you want to run.

1. Ensure "By Name" radio option is selected.
2. Use the "Search and select Scenarios" dropdown to find and click on the Scenarios you wish to add.
3. As you select them, they will appear below in the "Execution Order" section. This list displays the Scenario ID, name, and the number of associated test cases.
4. To change the execution sequence, simply click and hold the drag handle (⠿) next to a Scenario and move it to your desired position.
5. Once you are satisfied with your list, click "Done".

Selecting Scenarios by Filter

This approach is powerful when you want to dynamically run a group of Scenarios that share common attributes.

1. Select the "By Filter" option at the top.
2. A filter builder will appear. Click "+ Add Filter" to create your first rule.
3. For each rule, define your criteria by selecting:
   • A Field to filter on (e.g., Priority, Status, Scenario Name etc).
   • An Operator (e.g., contains, not contains).
   • A Value for the condition.
4. You can add multiple filter rows to create a precise and powerful selection logic. The suite will include all Scenarios that match all the defined filter conditions.
5. When your filters are set, click "Done".

Finalizing the Run

After clicking "Done" in the "Ad-hoc suite" window, you will be returned to the main Run modal. The Scenarios you selected or filtered will now be populated and ready for execution.

From here, the process is identical to any other test run:

• Test Case Selection: Review the selected Scenarios and choose to run all their associated test cases or a specific subset (by clicking the Test Case Count link).
• 
• Run Configuration: Set up all other run parameters, such as the OS, browser, host Agent, and any other job settings as you normally would.

Important Considerations

Understanding Version Control Behavior

It's crucial to understand how different run types interact with Version Control (VC) to ensure you are testing the intended logic. This is only applicable if you enabled Version Control for the current Project.

• Single Scenario Run: Executes your local version (checked-out version) of the Scenario and its related assets. This is ideal for testing your work-in-progress changes.
• Ad-hoc Suite Run: Behaves just like a single Scenario run. It executes your local, checked-out version of any assets in the suite.
• Test Suite Run: Executes the committed "system" version of all Scenarios and related Actions. This ensures that formal regression runs are consistent, stable, and reflect the official state of the project, not an individual's uncommitted changes.

Session Caching and Data Integrity

For your convenience, your Ad-hoc Suite configuration is cached in the user's session. The next time you open the Run modal, your last ad-hoc selection will be pre-populated.

The system also ensures data integrity automatically. If a Scenario or a Custom Field used in your last ad-hoc run has been deleted from the project, it will be silently removed from the cached definition to prevent errors on your next run.