Overview of sporadically failing test cases in Jenkins – UnitTH Jenkins plugin

Never, ever a stable blue build right? It does not seem to matter what you do, but for system integration tests there always seems to one or a few sporadically failing test cases. What can be extra annoying is that there might always be different test cases that sporadically fail, not the same one or two. To be able to determine the success of executed test suites there is a need to get an overview of the last few runs. The easiest way to visualise this is to create a matrix of test runs versus test cases to get a hit map with frequency of failures for specific test cases.

unitth-matrix

Test case failure spread.

For those using Jenkins there is a small Jenkins plugin at Sourceforge that can be added to the post build steps of any job that generates test results. The plugin gives you useful overview and stats for the available build test results.

Installation

Usage

  • Edit the configuration of a Jenkins job and in the post build section select the plugin
  • Run the job again to get the first matrix report, there will be a link in the sidebar
  • Every failed run of a test case has a link to the report trace in the build it was executed, simply click on the red hits in the matrix
Advertisements

3 Responses to “Overview of sporadically failing test cases in Jenkins – UnitTH Jenkins plugin”

  1. QA Says:

    When i configured this plugin and ran a Testng/Maven/Selenium job. It is showing the headers of the table but no information. Looking at jenkins logs there is no error in calculating the matrix. Which folder does this plugin look for results to calculate in this plugin?


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: