Data-driven soapUI JUnit integration with readable test reports

I have never really been satisfied with the way soapUI and JUnit integration had to be done. There where too many things that had to be maintained at two ends. This posting will describe an approach for runing soapUI test cases through JUnit using the data driven parts of JUnit with a small extension that makes the reporting really useful (traceable). In short, soapUI test case names are not needed any more, JUnit will pick up all non disabled test cases in a soapUI test suite and will execute and use the retrieved soapUI test case name in the report.

  • Minimal JUnit wrapping test classes
  • Superb reporting

Using

  • soapUI APIs – for running tests and retrieveing test cases
  • JUnit 4.8.x+ Parameterized

There are may examples on the web that explains how to wrap JUnit test cases around soapUI test cases. The basic approach for performing this is to simply map all test cases (test case names as strings) into single JUnit test cases. This is a very fragile approach since any change in a test case name needs to be reflected in the JUnit test suite.

  • soapUI test cases can be renamed at will
  • soapUI test cases can be enabled/disabled at will

Both of the above will break any JUnit test suites of not maintained properly.

JUnit 4 data-driven

JUnit 4.x comes with a feature where tests can be Parameterized or in other words data-driven. By using the Parameterized functionality it is rather trivial to extract all test cases to be executed and feed the test cases names as arguments to the test classes. This approach makes the JUnit test classes minimal and easy to maintain since all test case are read from the soapUI project files.

Sounds superb out of the box but things are seldom as fancy as the first appear. JUnit lacks a proper mechanism for the test reporting, all executed tests simply get names in the following format.


package.testclass
  \-<JUnit test case name>[0]
  \-<JUnit test case name>[1]
  \-<JUnit test case name>[2]
  \- ...

Not very useful since this makes the tracing back to the actual soapUI test case rather annoying.

So how to get what we want in place, starting from the end?

JUnit Parameterized extensions

I struggled with this bit for a while before I got it to work. After trying a few different approaches for extending the native JUnit functionality the easiest approach turned out to be to clone and extend the Parameterized class from the JUnit jar. Download any source jars from https://github.com/KentBeck/junit/downloads and fetch the org/junit/runners/Parameterized.java.

With the entire class code available the extension was trivial by simply edit the returned string from the getName and testName methods. I choose to use the exact name as given in the soapUI test suite, replacing all whitespace characters with an underscore ‘_’ sign. The soapUI test case name is fed through the JUnit class as described further down.


// My class is named ParameterizedExtended and is an exact clone of Parameterized apart from
// the changes below.

// Interface renaming to avoid any possible conflicts with the original classes in the JUnit diet
@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.METHOD)
public static @interface ParametersExtended {
}

@Override
protected String getName() {
  Object[] tcName = fParameterList.get(fParameterSetNumber);
  return String.format("[%s]-%s", fParameterSetNumber,((String)tcName[0]).replaceAll(" ", "_"));
}


@Override
protected String testName(final FrameworkMethod method) {
  Object[] tcName = fParameterList.get(fParameterSetNumber);
  return String.format("%s", ((String)tcName[0]).replaceAll(" ", "_"));
}

The JUnit test class

For enabling the data-driven parts the test class fist of all needs to use the @RunWith annotation pointing at your extended Parameterized class. Apart from this there is a minimum of methods to be implmented the CTOR, the method for getting all test cases, the actual test case and the soapUI runner.

Implementing the test class

Annotations used, @RunWith, “@ParametersExtended” (from the interface in the extended Parameterized class), @Test. Every test case gets an instance where a unique test case name is set based on the collection returned in the getTestCases method. The collection is really a set of String arrays but in this example only a single String is needed, the name of a soapUI test case.


@RunWith(ParameterizedExtended.class)
public class DemoTestSuite {

  private String testCaseName;
  public DemoTestSuite(String testCaseName) {
    super();
    this.testCaseName = testCaseName;
  }

  private boolean runSoapUITestCase(String testCase) {
    ...
  }

  @ParametersExtended
  public static Collection getTestCases() {
    ...
  }

  @Test
  public void TC_SOAP() {
    assertTrue(runSoapUITestCase(this.testCaseName));
  }
}

Fetching all the soapUI test cases

The getTestCases method looks for test cases in the soapUI project and returns all test cases present in a given test suite ignoring all disabled ones.


public static Collection<String[]> getTestCases() {
  final ArrayList<String[]> testCases = new ArrayList<String[]>();
  WsdlProject soapuiProject = new WsdlProject("PROJECT_FILE_NAME");
  WsdlTestSuite wsdlTestSuite = soapuiProject.getTestSuiteByName("TEST_SUITE_NAME");
  List<TestCase> testCaseStrings = wsdlTestSuite.getTestCaseList();

  for (TestCase ts : testCaseStrings) {
    if (!ts.isDisabled()) {
      testCases.add(new String[] {ts.getName()});
    }
  }
  return testCases;
}

Running the soapUI test case

A basic soapUI runner as described in any soapUI forums or tutorials, add it to your framework and kick off some tests.


public static boolean runSoapUITestCase(WsdlProject soapuiProject, String testSuite, String testCase) {
  TestRunner.Status exitValue = TestRunner.Status.INITIALIZED;
  WsdlProject soapuiProject = new WsdlProject("PROJECT_FILE_NAME");
  WsdlTestSuite wsdlTestSuite = soapuiProject.getTestSuiteByName(testSuite);
  if (wsdlTestSuite == null) {
    System.err.println("soapUI runner, test suite is null: "+testSuite);
    return false;
  }
  WsdlTestCase soapuiTestCase = wsdlTestSuite.getTestCaseByName(testCase);
  if (soapuiTestCase == null) {
    System.err.println("soapUI runner, test case is null: " + testCase);
    return false;
  }
  soapuiTestCase.setDiscardOkResults(true);
  WsdlTestCaseRunner runner = soapuiTestCase.run(new PropertiesMap(), false);
  exitValue = runner.getStatus();

  System.out.println("soapUI test case ended ('" + testSuite + "':'"+ testCase + "'): " + exitValue);
  if (exitValue == TestRunner.Status.FINISHED) {
    return true;
  } else {
    return false;
  }
}

The final output

With a minimal test class implementation, only the soapUI project file and test suite needs to be defined to get an easy to maintain JUnit/soapUI framework that has the kind of reporting one would expect to be in place.

package.testclass
  \-<soapUI test case name A>[0]
  \-<soapUI test case name B>[1]
  \-<soapUI test case name C>[2]
  \- ...

Advertisements

7 Responses to “Data-driven soapUI JUnit integration with readable test reports”

  1. reputation management Says:

    Wow that was unusual. I just wrote an extremely long comment but
    after I clicked submit my comment didn’t show up. Grrrr… well I’m not writing all
    that over again. Anyway, just wanted to say wonderful blog!

  2. Rao Says:

    Hi,
    Thank you for the detailed article. And i found it very interesting.
    https://github.com/KentBeck/junit/downloads this is not working, but, managed to get the jar file from https://github.com/mdm-releases/junit-releases.git.
    By following the article, created 2 classes
    1. ParameterizedExtended.java – with getName, testName methods as explained
    2. DemoTestSuite.java – with runSoapUITestCase, getTestCases, TC_SOAP methods as explained

    I am running into compilation errors for the following methods of ParameterizedExtended.java
    # How to provide fPrameterList, fParameterSetNumber? pass as parameters to getName method?

    @Override
    protected String getName() {
    Object[] tcName = fParameterList.get(fParameterSetNumber);
    return String.format("[%s]-%s", fParameterSetNumber,((String)tcName[0]).replaceAll(" ", "_"));
    }

    # And similarly in testName method.
    # Also compilation issue in runSoapUITestCase method where it is taking input of soapuiProject, and assigning

    WsdlProject soapuiProject = new WsdlProject("PROJECT_FILE_NAME");

    # @ParametersExtended – could not be found, should it be ParameterizedExtended?
    Am not a regular Java programmer. Appreaciate your help.

    • andnyb Says:

      Hi Rao, I updated the post a little since you made me realise that I was a bit unclear. Anyhow the ParametersExtended shall map to the interface in the Parameterized class you have ‘extended’ renamed or not.

      • Rao Says:

        Thank you for taking your time and reply, so kind of you. I will go throught it again, let you know how it goes.
        Regards,
        Rao.

  3. Guillaume Says:

    Hi,
    Thank you for the detailed information, it works fine. However, when I’m trying to execute the same procedure with SOAP UI Pro, I get the error: “java.lang.ClassCastException: com.eviware.soapui.impl.wsdl.WsdlProject cannot be cast to com.eviware.soapui.impl.wsdl.WsdlProjectPro”. I tried many ways to resolve this issue using Pro objects like WsdlProjectPro, WsdlTestSuitePro, SoapUIProTestCaseRunner but I’m still getting the same issue. So, do you have the SOAP UI Pro version of this procedure in order to fetch and to run all the soapUI Pro test cases?

    Thanks.

  4. kwadz Says:

    Hi,
    This is a very good idea and design but I can’t make it work. Could you provide a link to your full ParameterizedExtended.java ?


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: