Using Jmeter with Jenkins

The problem :

 

A performance problem was found on one pre-production platform while running manually Jmeter. This problem was found after the release was done which postpone our release.

 

The solution :

Performance tests should be done daily as part of the continuous integration. Thus we can discover problems of performance easily and fix earlier major problems.
Assuming you have already have a jmx file(Jmeter file), you need to call this file from maven with Jenkins . The second step is to parametrize variables in Jmeter in order to modify some variables from Jenkins ( very useful).

 

Call Maven from Jenkins

 

The configuration of the job Jenkins to call maven from Jenkins :


Configuration Build

 

Some variables have been parameterized such as NUMBER_ITERATION and NUMBER_USER . These Jenkins variables are being pass to maven variables such as number.user.jenkins and number.it.jenkins.

 

Therefore it is possible to modify those variables when launching Jmeter from Jenkins manually. The default values are used when the job is launched periodically.

 

 

I used the plugin “Publish Performance test result report” to report performance and detect bugs. There are other reporting tools in Jenkins for Jmeter.

 

Call Jmeter from Maven  to pass variables

 

This article was very useful to mavenify Jmeter : https://blog.codecentric.de/en/2014/01/automating-jmeter-tests-maven-jenkins/.

I chose the plugin jmeter-maven-plugin to do it. I used the same method as in the article.I just added commons-logging as dependency to make it work :

<profile>
   <id>my_profile/id>
        <properties>
                   <performancetest.webservice.host>localhost</performancetest.webservice.host>
         <performancetest.webservice.port>8080</performancetest.webservice.port>
         <performancetest.csv.file>data.csv</performancetest.csv.file>                                    
<performancetest.number>${number.it.jenkins}</performancetest.number>
         <number.user.profile>${number.user.jenkins}</number.user.profile>
       </properties
</profile>
    <build>
      <plugins>
          <plugin>
        <groupId>com.lazerycode.jmeter</groupId>
        <artifactId>jmeter-maven-plugin</artifactId>
        <version>1.8.1</version>
        <configuration>
          <testResultsTimestamp>false</testResultsTimestamp>
          <overrideRootLogLevel>DEBUG</overrideRootLogLevel>
          <suppressJMeterOutput>false</suppressJMeterOutput>
           <ignoreResultFailures>true</ignoreResultFailures>
          <testFilesIncluded>
                      <jMeterTestFile>myfile.jmx</jMeterTestFile>
           <propertiesUser>
    	       <number.it>${performancetest.number}</number.it>
               <webservice.host>${performancetest.webservice.host}</webservice.host>
               <webservice.port>${performancetest.webservice.port}</webservice.port>
               <csvfile>${performancetest.csvfile}</csvfile>
               <number.user>${number.user.profile}</number.user>
            </propertiesUser>
        </configuration>
           <dependencies>
          <dependency>
            <groupId>kg.apc</groupId>
            <artifactId>jmeter-plugins</artifactId>
            <version>1.0.0</version>
            <exclusions>
              <exclusion>
                  <groupId>kg.apc</groupId>
                  <artifactId>perfmon</artifactId>
              </exclusion>
              <exclusion>
                  <groupId>org.apache.jmeter</groupId>
                  <artifactId>jorphan</artifactId>
              </exclusion>
            </exclusions>
          </dependency>
        </dependencies>
      </plugin>
   </plugins>
   </build>
   <dependencies>
   <dependency>
   <groupId>commons-logging</groupId>
   <artifactId>commons-logging</artifactId>
   <version>4.0.6</version>
</dependency>

 

This list of variables are passed to the Jmx file :

 <propertiesUser>
    <number.it>${performancetest.number}</number.it>
               <webservice.host>${performancetest.webservice.host}</webservice.host>
               <webservice.port>${performancetest.webservice.port}</webservice.port>
           <csvfile>${performancetest.csvfile}</csvfile>
           <number.user>${number.user.profile}</number.user>
</propertiesUser>

Modify JMX file for parametrization

 

The final piece of work is to parametrize the actual JMX file. I found informations how to parametrize this here : http://stackoverflow.com/questions/12859085/set-jmeter-properties-using-maven-plugin.

 

After few tests I manage to find the exact wording to pass variables from maven to Jmeter. An example will tell more. For example you can define the number of thread and loop in “Thread Group”. Number of threads = ${__P(number.user,7)}. The default value is 7 if number.user is not passed to the JMX file.

 

Also here is the actual soap request with the url and port parametrised :

As you can see some variables are inside the request ${input1} and ${input2}. They are actually being fed from a CSV file. Tutorial : http://ivetetecedor.com/how-to-use-a-csv-file-with-jmeter/

 

In Jmeter, the CSV file is being loaded from “CSV Data Set Config”. Example of CSV configuration for Jmeter  :

Variable Names (comma-delimited) : input1,input2

FileName                                            : resources/${__P(csvfile,default.csv)}

FileName is where the csv file is located in my arborescence.

The CSV file contains the data used when calling the SOAP request. Example of CSV file :

data1;test1

data2;test2

data3;test3

 

At the first iteration, the first line of the CSV file will be used. Therefore a SOAP request will be send with input1=data1 and input2=test1. Then the next line will be used and so on .

 

 

TROUBLESHOOTING

 

At first I did not know why variables would not be pass to the JMX file from Jenkins. There was no errors in the logs “target\jmeter\logs” and the jtl file was not generated.I had to look in target\jmeter\bin to check that my variables were passed in the files.

 

My problem was that the names of my variables were different from the variables names in JMX. I resolved my issues by checking the user file.

 

User variables defined in maven as <propertiesUser> are passed in this file target\jmeter\bin\user.properties.

 

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s