Spec Files Explorer
Holistic performance view of your Spec Files
The Spec Files Explorer feature allows users to sort and analyze tests based on specific metrics to detect inconsistencies or challenging spec files that may require further examination. These metrics include:
- 1.Duration: The time taken for a spec file to execute
- 2.Failure Rate: The percentage of spec file executions that failed during the selected time
- 3.Timeout Rate: The percentage of a spec file executions that timed out during a specific time
- 4.Flakiness Rate: The percentage of a spec file executions that exhibited flakiness during the selected time
- 5.Suite Size: The number of tests in a spec file
This feature helps identify the most problematic spec files, such as:
- spec files with high flakiness
- long duration
- spec files with high failure rate
In addition to the metrics previously mentioned, the Explorer feature allows users to sort and filter spec files by tags, author, branch, and filename.
For the Duration metric, the tool provides an option to exclude or include failed executions in the list of spec files, which can be toggled with a flag at the top right of the list panel.
Spec Files Explorer
After choosing a particular spec file, users can analyze its behavior over time through six graphs that display the following metrics:
- 1.Average Duration: The average time executing a spec file.
- 2.Overall duration: The total time it takes to run a suite of tests from start to finish.
- 3.Suite size: The number of tests in the spec files.
- 4.Failure rate: The percentage of spec file executions that failed during the selected time.
- 5.Flakiness rate: The percentage of spec file executions that exhibited flakiness during the selected time.
- 6.Timeout Rate: The percentage of spec files timed out during a specific time.
Upon selecting a specific date from the timeline on one of the graphs, all the metrics will be displayed for that date. It allows users to examine the correlation between the metrics and any possible issues they may reveal. The Explorer also presents the execution history, where users can zoom into a specific period and analyze metrics related to test runs.
Zoom into one Spec File metrics
By providing these metrics and views, the Spec Files Explorer feature can help users identify patterns and trends in their execution history and make data-driven decisions to improve the reliability and efficiency of testing processes.
Last modified 9d ago