Build.xml for Selenium Webdriver + Ant + TestNg + XSLT (Automation Reporting Chapter 1)

Automation reporting has always been a topic which makes people scratch their head 🙂

From my view point automation reporting can be divided into 2 chapters:

Chapter 1: Ant + TestNg + XSLT concept

Suite reports is a mechanism which assist people in debugging their test scripts & also reflects the overall execution status of each suite.

Examples:

Problem in chapter 1:

Executing suites as TestNg is prety simple both via eclipse GUI or terminal. As all you need to do is “right-click on suite file > run as TestNg”. This report is only meant for each suite you execute. Output is stored in ‘test-output’ folder within your project directory.

The problem with this is procedure is that the content of ‘test-output’ folder is replaced each time you execute a suite. Resulting in lost of all history of test suites executed in past.

Solution:

Implement ‘Ant along with TestNg or XSLT’ concept to execute multiple suites together and store all your reports in different directories with ‘date& time stamp + suite name’ as directory name.

This can be controlled via build.xml file in Ant.

Chapter 2: Test Automation Dashboard

A dashboard which shows you a proper collection of all information required to take appropriate decisions on whether a build is ready to go live or not.

A complete collection of:

  • History of test suites executed & their results which can be accessed anytime based on date filters
  • Real-time test execution status for ‘N’ number of test suites

=============================================

Focus of this blog post is on providing the solution for Chapter 1 only. Details on Chapter 2 will be shared later.

Now, we know that Ant ‘build.xml’ controls the behavior of how we execute our test suites from terminal.

There are 2 ways to implement build.xml

Case 1: Single build.xml only

Advantages:

In order to overcome the problem mentioned in ‘Chapter1’, I have configured the ‘build.xml’ which has following advantages:

  • Stores ‘TestNg’ or ‘XSLT’ in seperate directories for each suite as per ‘Date & Time stamp + Suite name’ as directory name. Example format: yyyy-mm-dd_hh-mm-ss Am/Pm_SuiteName
  • Also, configured to execute ‘N’ number of test suites together within same build.xml

Build.xml content:

In this case ‘build.xml’ is placed under the root directory of project & the content of build.xml is as follows:

<?xml version="1.0" encoding="UTF-8"?>
<project name="projectName" default="clean" basedir="."> <!-- dot indicates that basedir is pointing towards project root directory -->

 <!-- ========== Initialize Properties =================================== -->
 <property name="ws.home" value="${basedir}" />
 <property name="ws.jars" value="/Users/Yash/Downloads/Automation/jars/new/allJars" />
 <property name="test.dest" value="${ws.home}/build"/>
 <property name="test.src" value="${ws.home}/src"/>
 <property name="ng.result" value="test-output"/>

 <echo> value of base dir = ${basedir} </echo>

 <!-- properties for copying the results -->
 <tstamp>
 <format property="year" pattern="yyyy" />
 <format property="DSTAMP" pattern="yyyy-MM-dd" />
 <format property="TSTAMP" pattern="HH:mm:ss" />
 <format property="dateversion" pattern="yyyy.MM.dd.HH:mm:ss" />
 <format property="time.stamp" pattern="yyyy-MM-dd_HH-mm-ss aa_"/>
 </tstamp>

 <property name="testng.report.dir" value="${ws.home}/testngReports/${time.stamp}"/>
 <property name="testngXslt.report.dir" value="${ws.home}/testngXsltReports/${time.stamp}"/>

 <!-- ====== For setting the classpath ==== -->
 <target name="setClassPath" unless="test.classpath">
 <path id="classpath_jars">
 <fileset dir="${ws.jars}" includes = "*.jar"/>
 </path>
 <pathconvert pathsep=":"
 property="test.classpath"
 refid="classpath_jars" />

 </target>

 <!-- ============ Initializing other stuff =========== -->
 <target name="init" depends="setClassPath">
 <tstamp>
 <format property="start.time" pattern="MM-dd-yyyy (HH-mm-ss)"/>
 </tstamp>
 <condition property="ANT"
 value="$(env.ANT_HOME)/bin/ant.bat"
 else="$(env.ANT_HOME)/bin/ant">
 <!-- os family="windows" /-->
 <os family="mac" />
 </condition>

 <taskdef name="testng" classpath="/Users/Yash/Downloads/Automation/jars/new/allJars/testng-6.8.5.jar" classname="org.testng.TestNGAntTask" /> 

 <!--
 <!- use direct path for classpath if you don't prefer to use 'lib' directory: classpath="/Users/yash/Documents/Jar Files/testng-6.8.jar" ->
 <taskdef name="testng" classname="org.testng.TestNGAntTask">
 <classpath>
 <pathelement location="./lib/testng.jar"/>
 </classpath>
 </taskdef>
 -->

 </target>

 <target name="all">
 </target>

 <!-- cleaning the destination folders -->
 <target name="clean">
 <echo message="deleting existing build directory"/>
 <delete dir="${test.dest}"/>
 </target>

 <!-- target for compiling the java files -->
 <target name="compile" depends="init, clean">
 <delete includeemptydirs="true" quiet="true">
 <fileset dir="${test.dest}" includes="**/*"/>
 </delete>
 <echo message="making directory....."/>
 <mkdir dir="${test.dest}"/>
 <echo message="classpath-------: ${test.classpath}"/>
 <echo message="compiling....."/>
 <javac
 debug="true"
 destdir="${test.dest}"
 srcdir="${test.src}"
 classpath="${test.classpath}"
 includeantruntime="true"/>
 </target>

 <!-- build -->
 <target name="build" depends="init">
 </target>

 <!-- ========== Test executions & Generating reports using Testng utility for multiple suites ============== -->
 <!-- run -->
 <target name="run" depends="compile">

 <!--suite 1 begin -->
 <property name="suite.web.CopyRegressionCustomer" value="CopyRegressionCustomer" />
 <testng classpath="${test.classpath}:${test.dest}" suitename="CopyRegressionCustomer" outputDir="test-output" >
 <xmlfileset dir="." includes="webSuites/CopyRegressionCustomer.xml" />
 </testng>
 <antcall target="testngReportCopy">
 <param name="testngDir" value="${testng.report.dir}${suite.web.CopyRegressionCustomer}"/>
 </antcall> 

 <!--suite 2 begin -->
 <property name="suite.web.Copy2RegressionCustomer" value="Copy2RegressionCustomer" />
 <testng classpath="${test.classpath}:${test.dest}" suitename="Copy2RegressionCustomer" outputDir="test-output" >
 <xmlfileset dir="." includes="webSuites/Copy2RegressionCustomer.xml" />
 </testng>
 <antcall target="testngReportCopy">
 <param name="testngDir" value="${testng.report.dir}${suite.web.Copy2RegressionCustomer}"/>
 </antcall> 

 </target>

 <target name="testngReportCopyAndReportParser">
 <!-- Copy to TestNG report directory-->
 <mkdir dir="${testngDir}">
 </mkdir>

 <!-- to copy previous suite result to TestNG report directory -->
 <copy todir="${testngDir}">
 <fileset dir="test-output"/>
 </copy> <!-- end of Testng Report -->
 </target>

 <!-- ========== Test executions & Generating reports using Testng XSLT utility for multiple suites ============== -->
 <!-- run -->
 <target name="runAsTestngXslt" depends="compile">

 <!--suite 1 begin -->
 <property name="suite.web.CopyRegressionCustomer" value="CopyRegressionCustomer" />
 <testng classpath="${test.classpath}:${test.dest}" suitename="CopyRegressionCustomer" outputDir="test-output" >
 <xmlfileset dir="." includes="webSuites/CopyRegressionCustomer.xml" />
 </testng>
 <antcall target="testngXsltReportCopy">
 <param name="testngXsltDir" value="${testngXslt.report.dir}${suite.web.CopyRegressionCustomer}"/>
 </antcall> 

 <!--suite 2 begin -->
 <property name="suite.web.Copy2RegressionCustomer" value="Copy2RegressionCustomer" />
 <testng classpath="${test.classpath}:${test.dest}" suitename="Copy2RegressionCustomer" outputDir="test-output" >
 <xmlfileset dir="." includes="webSuites/Copy2RegressionCustomer.xml" />
 </testng>
 <antcall target="testngXsltReportCopy">
 <param name="testngXsltDir" value="${testngXslt.report.dir}${suite.web.Copy2RegressionCustomer}"/>
 </antcall> 

 </target>

 <target name="testngXsltReportCopyAndReportParser">
 <!-- Copy to TestNG report directory-->
 <mkdir dir="${testngXsltDir}">
 </mkdir>

 <!-- to copy previous suite result to TestngXslt report folder -->
 <xslt in="${ws.home}/test-output/testng-results.xml" style="src/xslt/testng-results.xsl"
 out="${testngXsltDir}/index.html" classpathref="test.c" processor="SaxonLiaison">

 <param name="testNgXslt.outputDir" expression="${testngXsltDir}" />
 <param name="testNgXslt.showRuntimeTotals" expression="true" />
 <param name="testNgXslt.sortTestCaseLinks" expression="true" />
 <param name="testNgXslt.testDetailsFilter" expression="FAIL,SKIP,PASS,CONF,BY_CLASS" />
 </xslt> <!-- end of TestngXslt Report -->
 </target>

 <path id="test.c">
 <fileset dir="${ws.jars}" includes="*.jar">
 <include name="mail.jar"/>
 <include name="activation-1.0.2.jar"/>
 </fileset>
 </path>

 <!-- ========== Generating reports using Testng XSLT utility for single suite only ============== -->
 <target name="report" depends="run">
 <!-- TestngXslt report -->
 <mkdir dir="${testngXslt.report.dir}">
 </mkdir>

 <!-- to copy previous suite result to TestngXslt report folder -->
 <xslt in="${ws.home}/test-output/testng-results.xml" style="src/xslt/testng-results.xsl"
 out="${testngXslt.report.dir}/index.html" classpathref="test.c" processor="SaxonLiaison">

 <param name="testNgXslt.outputDir" expression="${testngXslt.report.dir}" />
 <param name="testNgXslt.showRuntimeTotals" expression="true" />
 <param name="testNgXslt.sortTestCaseLinks" expression="true" />
 <param name="testNgXslt.testDetailsFilter" expression="FAIL,SKIP,PASS,CONF,BY_CLASS" />
 </xslt> <!-- end of TestngXslt Report -->
 </target>

 <target name="RunAndViewReport" depends="report">
 <exec executable="${browser}" spawn="yes">
 <arg line="'file:///${testngXslt.report.dir}/index.html'" />
 </exec>
 </target>

 <target name="sendMail" depends="RunAndViewReport">
 <zip destfile="${testngXslt.report.dir}/Report.zip" basedir="${testngXslt.report.dir}"/>
 <mail mailhost="smtp.gmail.com" mailport="465" subject="Notification of TESTNG build result" ssl="false" user="automationmoolya@gmail.com" password="moolya@universe">
 <from address="automationmoolya@gmail.com"/>
 <to address="yagnesh@moolya.com"/>
 <message>The build has finished. A details report of this build is attached</message>
 <attachments>
 <fileset dir="testngXslt.report.dir">
 <include name="**/*.zip"/>
 </fileset>
 </attachments>
 </mail>
 </target>

 <target name="install-jars" description="Install ANT optional jars">
 <get dest="${ws.home}/lib/mail.jar" src="file:///${ws.home}/lib/mail.jar"/>
 <fileset dir="${ws.jars}" includes="*.jar">
 <include name="mail.jar"/>
 <include name="activation-1.0.2.jar"/>
 </fileset>
 </target>

</project>

How to execute:

step 1: Check if ‘Ant’ is working or not:

Make sure ant path is set appropriately in system environment variables. we can test this by typing ‘ant’ in terminal & it should display something like this if all settings are correct:

Buildfile: build.xml does not exist!
Build failed

step 2: Execute test suites :: To generate report in TestNg format:

Now in terminal, navigate to the project directory which has your ‘build.xml’. Then run following command:

Yash@YASH-PC ~
$ cd git/sugarCRM/SugarCRM/
Yash@YASH-PC ~/git/sugarCRM/SugarCRM (master)
$ ant run

That’s it…This would run all test suites specified in build.xml

Note 1: ‘run’ in the above command is the ‘target'(ie, method/function in terms of java) name from ‘build.xml’ which executes the statements of those target.

Note 2: ‘run’ target depends on ‘compile’ target which in turn depends on ‘init’ & ‘clean’ target. Hence, the only command we need to run from terminal is ‘ant run’ from project directory, which in turn will also take care of init, clean, compile process.

(or) step 2: Execute test suites :: To generate report in XSLT format:

Instead of executing ‘run’ target, execute ‘runAsTestngXslt’ target. Done 🙂

Yash@YASH-PC ~
$ cd git/sugarCRM/SugarCRM/
Yash@YASH-PC ~/git/sugarCRM/SugarCRM (master)
$ ant runAsTestngXslt

 

Case 2: Multiple build.xml’s

Sometimes total number of test suites are too much to be executed all together (i.e, serial execution of test suites). Sometimes, based on our sub-products we wish to execute test suites separately & do not wish to configure single build.xml for all test suites.

In such case, we can opt for multiple build.xml files where each one is targeted to execute specific set of test suites based on sub-products or our specific context.

Hence, this results in multiple build.xml files out of which one has to be a parent build.xml which in turn can call our individual child build.xml one after another.

Note: We may directly execute child build.xml from terminal but generally it is a door towards issues which are pertaining to identifying appropriate directory path for all the variables declared within the file. As per the ‘Ant’ developers it is recommended that you have a build.xml in root of project directory. Therefore we can have one parent build.xml in root directory which in turn takes care of calling child build.xml from sub-directories.

Build.xml content:

In this case, parent build.xml has to be placed in the ‘root’ of project directory system. All Child build.xml can be stored under single folder like ‘masterBuildFiles’. We may also rename the child build.xml based on our context.

Content of Parent build.xml:

<?xml version="1.0" encoding="UTF-8"?>
<project name="novopayBoi" default="clean" basedir=".">

 <!-- cleaning the destination folders -->
 <target name="clean">
 <echo message="deleting existing build directory"/>
 <delete dir="${test.dest}"/>
 </target>

 <target name="run">
 <!-- include any number of child build.xml as you wish-->
 <ant antfile="masterBuildFiles/buildXYZ_Template.xml" target="run" />
 <ant antfile="masterBuildFiles/build1_WebSanity.xml" target="run" />
 <ant antfile="masterBuildFiles/build2_WebRegression.xml" target="run" />
 </target>

</project>

Content of each Child build.xml: (Note: for each child build.xml we need to configure appropriately which suites to be executed within target ‘run’)

Note: The only difference between this child build.xml & the build.xml from case 1 above is the reference to ‘basedir’:

<project name="projectName" default="clean" basedir="./../../"> <!-- 2 directories back from child build.xml's current directory which is pointing to root directory -->

Remaining part is the same as single build.xml code sample from case 1.

That’s all it takes to generate test automation suite reports in TestNg or XSLT format.

Happy Testing!!

 

Advertisements

17 thoughts on “Build.xml for Selenium Webdriver + Ant + TestNg + XSLT (Automation Reporting Chapter 1)

  1. i am trying to mail report from build.xml but getting error:
    “Failed to send email: javax.mail.internet.MimeMessage”
    i had put activation.jar and mail.jar in ant’s lib folder

    1. Hello Neha,

      build.xml code in this blog doesn’t show how to send a automated mail.
      Question: Do you have a code ready for emailing reports & attachment?

      Try referring following code:
      step 1. First you need to have a Email code in your “Utils”. You may copy the code from
      https://github.com/YagneshShah/SeleniumLearnExploreContribute/blob/dev/orangeHRM/src/utils/SmsEmailNotification.java

      Note: this email code will refer variables from “config.properties” file in project root. Here are those properties for your reference:
      ############### Email properties ####################
      emailProjectName=projectName Release XYZ
      emailFromUser=abc@gmail.com
      emailFromPassword=xyz
      emailTo1=abc@gmail.com
      emailTo2=xyz@gmail.com
      emailBodyMessage=PFA Emailable Test Report & XML file for details.
      emailAttachment1Path=../orangeHRM/test-output/emailable-report.html
      emailAttachment2Path=../orangeHRM/test-output/testng-results.xml

      step 2. Use build.xml for sending automated emails via invoking Email code
      Add following snippet to your build.xml file within “testngReportCopyAndReportParser” target:

      or refer this link for complete updated version of build.xml, but be carefull in referring only those targets which are required by you:
      https://github.com/YagneshShah/SeleniumLearnExploreContribute/blob/dev/orangeHRM/masterBuildFiles/build2_WebRegression.xml

      1. No “Moolya’s Test Automation Dashboard” doesn’t support Junit report format.
        In our architecture & Project, we didn’t have requirement for Junit, therefore TestNg format is the only one supported for now.

        Well, it’s a open source & everyone is welcomed for their contribution. May be you can contribute a Junit parser 🙂

  2. Hi, I am fairly new to automation and trying to learn. In order to generate xslt reports i found the build.xlm from you thanks for that. Now i guess we need to to put the code for xslt reports to should be put on “src” folder of project. Can you share that code please ?

      1. Hi Yagnesh,

        I have two environments in my project hence I have two properties file,how I can be able to pass these filename in ant at run time.Help is highly appreciated.
        File names that I want to pass are
        a.Stage.properties
        b.prod.properties

        In the above property file I have defined data as key value pair,this file is basically an input file for me to read data based on environment.I have one more file called read.java ,from this class file,iam changing my property file now manually.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s