Advanced Analysis for Individual Tests
By providing these metrics, the Tests Explorer feature can help users identify patterns and trends in their test execution history and make data-driven decisions to improve the reliability and efficiency of testing processes.
Zoom into one Test metrics
Upon selecting a specific date from the timeline on one of the graphs, all the metrics will be displayed for that date. It allows users to examine the correlation between the metrics and any possible issues they may reveal.
For example, let's examine the test analyzed in the video above. During the analysis, it is observed a notable peak in the Average Duration of this test, which coincided with the highest Failure Rate recorded on the same day. This situation suggests a potential relationship between the duration of the test executions and the occurrence of failures.
The Executions History feature displays tests histograms with data for a specific time. Users can interact with the histograms using a brush tool, allowing them to zoom in and focus on a particular period of interest.
Users can also choose to organize the executions by branch using the "Collapse branch" flag. This allows for a more organized view of the data, grouped by the branches.
The Passed Duration histogram displays the distribution of durations for the test executions that have passed. It provides insights into the time taken for successful test runs. This information can be valuable in understanding the typical duration for successful tests and identifying any outliers or patterns that may require further investigation.
The Non-Flaky Duration histogram represents the durations of test executions that do not exhibit flakiness. This histogram focuses on the durations of executions that consistently yield reliable and consistent results. It helps understand the average time taken for non-flaky test executions and provides a baseline for comparison with the overall duration histogram.
The Tests Execution History feature organizes every test execution in a timeline format, where colors represent their respective execution status (Failed, Passed, and Flaky) that the users can filter. Each test execution entry provides users with essential information such as the date, time, duration, branch, commit message, and author. Additionally, for failed tests, the feature presents an error preview.
Moreover, the feature presents the top errors encountered and the Executions History, where users can zoom into a specific period and analyze various metrics related to their runs.
Executions History - Tests
In general, the Tests Execution History feature empowers users with a timely analysis of the metrics and executions linked to each test, resulting in improved testing practices and more efficient software development processes.