ScalaTests on Jenkins using Maven, use case examples

This is a post for how to do it with Maven, the same functionality can be acheived by using Sbt if preferring that option.

There are a number of threads out there that addresses the topic of configuring and running ScalaTest test cases/specs on Jenkins, some threads older than others but since this seems to be an re-occuring question topic why not to examplify how it can be done based for some of the most common use cases that seems to be popping up in the forums. So if you have problems setting up jobs that are running specific test specs or test that are tagged then have a look at the examples below.

Core knowledge basics

scalatest-maven-plugin

This plugins enbles execution of ScalaTests tests using Maven without any extra fuzz like using @RunWith(classOf[JUnitRunner])… I takes a set of configuration options that are well defined on the . So start by adding this plugin to your Maven pom build section.

The ScalaTest options used in the examples below are added to the plugin’s excecution configuration according to.

<plugin>
   <groupId>org.scalatest</groupId>
   <artifactId>scalatest-maven-plugin</artifactId>
   <version>1.0-M2</version>
   <configuration>
      <reportsDirectory>${project.build.directory}/scalatest-reports</reportsDirectory>
      <junitxml>.</junitxml>
      <testFailureIgnore>true</testFailureIgnore>
      <filereports>WDF TestSuite.txt</filereports>
      <forkMode>never</forkMode>
      <parallel>${runSpecsInParallel}</parallel>
   </configuration>
   <executions>
      <execution>
         <id>test</id>
         <goals>
            <goal>test</goal>
         </goals>
         <configuration>
            <membersOnlySuites>${membersOnlySuites}</membersOnlySuites>
            <suites>${suites}</suites>
            <tagsToExclude>${tagsToExclude}</tagsToExclude>
            <tagsToInclude>${tagsToInclude}</tagsToInclude>
            <wildcardSuites>${wildcardSuites}</wildcardSuites>
         </configuration>
      </execution>
   </executions>
</plugin>

maven-profiles

For the majority of the use cases below we are using Maven project profiles the are piped to the Maven execution using the -P options. By example: mvn test -Pmyprofile1

The profiles we are setting up are defining NONE or more of the ScalaTest configuration options described above.

<profile>
   <id>myprofile1</id>
   <properties>
      <tagsToExclude>SlowTest</tagsToExclude>
      <membersOnlySuites>com.mycompany.app.api</membersOnlySuites>
   </properties>
</profile>

Jenkins jobs

These are straight forward, create a maven job that runs a goal in line with: mvn test -Pmyprofile1.

Use cases addressed

1. Running a selected set of test specs based on a package structure using, membersOnlySuites and wildcardSuites
2. Running a selected set if tests based on spec names using suites
3. Running a selected set of tests that are tagged using tagsToInclude
4. Running a selected set of tests but exluding tests that have a certain tag
5. Running tests with a given profile and overriding scalatest properties

Assuming the following test spec structure for all examples below

|- com.mycompany.app.api.v1
| |- CreateEntitySpec.scala
| \- DeleteEntitySpec.scala
\- com.mycompany.app.api.v2


Selected set of tests based on package structure

The membersOnlySuites picks up specs that are directly placed in the given package. It does not pick up any specs in its sub-packages.

// Runs all v1 tests
mvn test -DmembersOnlySuites=com.mycompany.app.api.v1

// Runs all v1 and v2 tests
mvn test -DmembersOnlySuites=com.mycompany.app.api.v1,com.mycompany.app.api.v2

The property wildcardSuites on the other hand will pick up all specs in the given package and all sub packages.

// Runs all v1 and v2 tests
mvn test -DwildcardSuites=com.mycompany.app.api

 

Running a selected set if tests based on spec names

// Runs all tests in the CreateEntitySpec and DeleteEntitySpec
mvn test -Dsuites=com.mycompany.app.api.v1.CreateEntitySpec,com.mycompany.app.api.v1.DeleteEntitySpec

 

Running a selected set of tests that are tagged

// Runs all tests in the CreateEntitySpec and DeleteEntitySpec that are tagged as SmokeTest and/or FastTest
mvn test -Dsuites=com.mycompany.app.api.v1.CreateEntitySpec,com.mycompany.app.api.v1.DeleteEntitySpec
-DtagsToInclude=SmokeTest,FastTest

 

Running a selected set of tests but exluding tests that have a certain tag

// Runs all test cases in all test spec but not those tests that are tagged as VerySlowTest
mvn test -DmembersOnlySuites=com.mycompany.app.api -DtagsToExclude=VerySlowTest

 

Running tests with a given profile and overriding scalatest properties

Resetting properties are done using the value None otherwise just overwrite by giving it a new value.

// Runs all tests as given in the myprofile1 profile in the pom file resetting
// the tagsToExclude property set in the profile in the pom.
mvn test -Pmyprofile1 -DtagsToExclude=None