Build.xml for Selenium Webdriver + Ant + TestNg + XSLT (Automation Reporting Chapter 1)

Automation reporting has always been a topic which makes people scratch their head 🙂

From my view point automation reporting can be divided into 2 chapters:

Chapter 1: Ant + TestNg + XSLT concept

Suite reports is a mechanism which assist people in debugging their test scripts & also reflects the overall execution status of each suite.

Examples:

• TestNg Report
• TestNg XSLT Report Overview
• TestNG XSLT Report Detailed
• Maven Report

Problem in chapter 1:

Executing suites as TestNg is prety simple both via eclipse GUI or terminal. As all you need to do is “right-click on suite file > run as TestNg”. This report is only meant for each suite you execute. Output is stored in ‘test-output’ folder within your project directory.

The problem with this is procedure is that the content of ‘test-output’ folder is replaced each time you execute a suite. Resulting in lost of all history of test suites executed in past.

Solution:

Implement ‘Ant along with TestNg or XSLT’ concept to execute multiple suites together and store all your reports in different directories with ‘date& time stamp + suite name’ as directory name.

This can be controlled via build.xml file in Ant.

Chapter 2: Test Automation Dashboard

A dashboard which shows you a proper collection of all information required to take appropriate decisions on whether a build is ready to go live or not.

A complete collection of:

• History of test suites executed & their results which can be accessed anytime based on date filters
• Real-time test execution status for ‘N’ number of test suites

=============================================

Focus of this blog post is on providing the solution for Chapter 1 only. Details on Chapter 2 will be shared later.

Now, we know that Ant ‘build.xml’ controls the behavior of how we execute our test suites from terminal.

There are 2 ways to implement build.xml

Case 1: Single build.xml only

Advantages:

In order to overcome the problem mentioned in ‘Chapter1’, I have configured the ‘build.xml’ which has following advantages:

• Stores ‘TestNg’ or ‘XSLT’ in seperate directories for each suite as per ‘Date & Time stamp + Suite name’ as directory name. Example format: yyyy-mm-dd_hh-mm-ss Am/Pm_SuiteName
• Also, configured to execute ‘N’ number of test suites together within same build.xml

Build.xml content:

In this case ‘build.xml’ is placed under the root directory of project & the content of build.xml is as follows:

<?xml version="1.0" encoding="UTF-8"?>
<project name="projectName" default="clean" basedir="."> <!-- dot indicates that basedir is pointing towards project root directory -->

 <!-- ========== Initialize Properties =================================== -->
 <property name="ws.home" value="${basedir}" />
 <property name="ws.jars" value="/Users/Yash/Downloads/Automation/jars/new/allJars" />
 <property name="test.dest" value="${ws.home}/build"/>
 <property name="test.src" value="${ws.home}/src"/>
 <property name="ng.result" value="test-output"/>

 <echo> value of base dir = ${basedir} </echo>

 <!-- properties for copying the results -->
 <tstamp>
 <format property="year" pattern="yyyy" />
 <format property="DSTAMP" pattern="yyyy-MM-dd" />
 <format property="TSTAMP" pattern="HH:mm:ss" />
 <format property="dateversion" pattern="yyyy.MM.dd.HH:mm:ss" />
 <format property="time.stamp" pattern="yyyy-MM-dd_HH-mm-ss aa_"/>
 </tstamp>

 <property name="testng.report.dir" value="${ws.home}/testngReports/${time.stamp}"/>
 <property name="testngXslt.report.dir" value="${ws.home}/testngXsltReports/${time.stamp}"/>

 <!-- ====== For setting the classpath ==== -->
 <target name="setClassPath" unless="test.classpath">
 <path id="classpath_jars">
 <fileset dir="${ws.jars}" includes = "*.jar"/>
 </path>
 <pathconvert pathsep=":"
 property="test.classpath"
 refid="classpath_jars" />

 </target>

 <!-- ============ Initializing other stuff =========== -->
 <target name="init" depends="setClassPath">
 <tstamp>
 <format property="start.time" pattern="MM-dd-yyyy (HH-mm-ss)"/>
 </tstamp>
 <condition property="ANT"
 value="$(env.ANT_HOME)/bin/ant.bat"
 else="$(env.ANT_HOME)/bin/ant">
 <!-- os family="windows" /-->
 <os family="mac" />
 </condition>

 <taskdef name="testng" classpath="/Users/Yash/Downloads/Automation/jars/new/allJars/testng-6.8.5.jar" classname="org.testng.TestNGAntTask" /> 

 <!--
 <!- use direct path for classpath if you don't prefer to use 'lib' directory: classpath="/Users/yash/Documents/Jar Files/testng-6.8.jar" ->
 <taskdef name="testng" classname="org.testng.TestNGAntTask">
 <classpath>
 <pathelement location="./lib/testng.jar"/>
 </classpath>
 </taskdef>
 -->

 </target>

 <target name="all">
 </target>

 <!-- cleaning the destination folders -->
 <target name="clean">
 <echo message="deleting existing build directory"/>
 <delete dir="${test.dest}"/>
 </target>

 <!-- target for compiling the java files -->
 <target name="compile" depends="init, clean">
 <delete includeemptydirs="true" quiet="true">
 <fileset dir="${test.dest}" includes="**/*"/>
 </delete>
 <echo message="making directory....."/>
 <mkdir dir="${test.dest}"/>
 <echo message="classpath-------: ${test.classpath}"/>
 <echo message="compiling....."/>
 <javac
 debug="true"
 destdir="${test.dest}"
 srcdir="${test.src}"
 classpath="${test.classpath}"
 includeantruntime="true"/>
 </target>

 <!-- build -->
 <target name="build" depends="init">
 </target>

 <!-- ========== Test executions & Generating reports using Testng utility for multiple suites ============== -->
 <!-- run -->
 <target name="run" depends="compile">

 <!--suite 1 begin -->
 <property name="suite.web.CopyRegressionCustomer" value="CopyRegressionCustomer" />
 <testng classpath="${test.classpath}:${test.dest}" suitename="CopyRegressionCustomer" outputDir="test-output" >
 <xmlfileset dir="." includes="webSuites/CopyRegressionCustomer.xml" />
 </testng>
 <antcall target="testngReportCopy">
 <param name="testngDir" value="${testng.report.dir}${suite.web.CopyRegressionCustomer}"/>
 </antcall> 

 <!--suite 2 begin -->
 <property name="suite.web.Copy2RegressionCustomer" value="Copy2RegressionCustomer" />
 <testng classpath="${test.classpath}:${test.dest}" suitename="Copy2RegressionCustomer" outputDir="test-output" >
 <xmlfileset dir="." includes="webSuites/Copy2RegressionCustomer.xml" />
 </testng>
 <antcall target="testngReportCopy">
 <param name="testngDir" value="${testng.report.dir}${suite.web.Copy2RegressionCustomer}"/>
 </antcall> 

 </target>

 <target name="testngReportCopyAndReportParser">
 <!-- Copy to TestNG report directory-->
 <mkdir dir="${testngDir}">
 </mkdir>

 <!-- to copy previous suite result to TestNG report directory -->
 <copy todir="${testngDir}">
 <fileset dir="test-output"/>
 </copy> <!-- end of Testng Report -->
 </target>

 <!-- ========== Test executions & Generating reports using Testng XSLT utility for multiple suites ============== -->
 <!-- run -->
 <target name="runAsTestngXslt" depends="compile">

 <!--suite 1 begin -->
 <property name="suite.web.CopyRegressionCustomer" value="CopyRegressionCustomer" />
 <testng classpath="${test.classpath}:${test.dest}" suitename="CopyRegressionCustomer" outputDir="test-output" >
 <xmlfileset dir="." includes="webSuites/CopyRegressionCustomer.xml" />
 </testng>
 <antcall target="testngXsltReportCopy">
 <param name="testngXsltDir" value="${testngXslt.report.dir}${suite.web.CopyRegressionCustomer}"/>
 </antcall> 

 <!--suite 2 begin -->
 <property name="suite.web.Copy2RegressionCustomer" value="Copy2RegressionCustomer" />
 <testng classpath="${test.classpath}:${test.dest}" suitename="Copy2RegressionCustomer" outputDir="test-output" >
 <xmlfileset dir="." includes="webSuites/Copy2RegressionCustomer.xml" />
 </testng>
 <antcall target="testngXsltReportCopy">
 <param name="testngXsltDir" value="${testngXslt.report.dir}${suite.web.Copy2RegressionCustomer}"/>
 </antcall> 

 </target>

 <target name="testngXsltReportCopyAndReportParser">
 <!-- Copy to TestNG report directory-->
 <mkdir dir="${testngXsltDir}">
 </mkdir>

 <!-- to copy previous suite result to TestngXslt report folder -->
 <xslt in="${ws.home}/test-output/testng-results.xml" style="src/xslt/testng-results.xsl"
 out="${testngXsltDir}/index.html" classpathref="test.c" processor="SaxonLiaison">

 <param name="testNgXslt.outputDir" expression="${testngXsltDir}" />
 <param name="testNgXslt.showRuntimeTotals" expression="true" />
 <param name="testNgXslt.sortTestCaseLinks" expression="true" />
 <param name="testNgXslt.testDetailsFilter" expression="FAIL,SKIP,PASS,CONF,BY_CLASS" />
 </xslt> <!-- end of TestngXslt Report -->
 </target>

 <path id="test.c">
 <fileset dir="${ws.jars}" includes="*.jar">
 <include name="mail.jar"/>
 <include name="activation-1.0.2.jar"/>
 </fileset>
 </path>

 <!-- ========== Generating reports using Testng XSLT utility for single suite only ============== -->
 <target name="report" depends="run">
 <!-- TestngXslt report -->
 <mkdir dir="${testngXslt.report.dir}">
 </mkdir>

 <!-- to copy previous suite result to TestngXslt report folder -->
 <xslt in="${ws.home}/test-output/testng-results.xml" style="src/xslt/testng-results.xsl"
 out="${testngXslt.report.dir}/index.html" classpathref="test.c" processor="SaxonLiaison">

 <param name="testNgXslt.outputDir" expression="${testngXslt.report.dir}" />
 <param name="testNgXslt.showRuntimeTotals" expression="true" />
 <param name="testNgXslt.sortTestCaseLinks" expression="true" />
 <param name="testNgXslt.testDetailsFilter" expression="FAIL,SKIP,PASS,CONF,BY_CLASS" />
 </xslt> <!-- end of TestngXslt Report -->
 </target>

 <target name="RunAndViewReport" depends="report">
 <exec executable="${browser}" spawn="yes">
 <arg line="'file:///${testngXslt.report.dir}/index.html'" />
 </exec>
 </target>

 <target name="sendMail" depends="RunAndViewReport">
 <zip destfile="${testngXslt.report.dir}/Report.zip" basedir="${testngXslt.report.dir}"/>
 <mail mailhost="smtp.gmail.com" mailport="465" subject="Notification of TESTNG build result" ssl="false" user="automationmoolya@gmail.com" password="moolya@universe">
 <from address="automationmoolya@gmail.com"/>
 <to address="yagnesh@moolya.com"/>
 <message>The build has finished. A details report of this build is attached</message>
 <attachments>
 <fileset dir="testngXslt.report.dir">
 <include name="**/*.zip"/>
 </fileset>
 </attachments>
 </mail>
 </target>

 <target name="install-jars" description="Install ANT optional jars">
 <get dest="${ws.home}/lib/mail.jar" src="file:///${ws.home}/lib/mail.jar"/>
 <fileset dir="${ws.jars}" includes="*.jar">
 <include name="mail.jar"/>
 <include name="activation-1.0.2.jar"/>
 </fileset>
 </target>

</project>

How to execute:

step 1: Check if ‘Ant’ is working or not:

Make sure ant path is set appropriately in system environment variables. we can test this by typing ‘ant’ in terminal & it should display something like this if all settings are correct:

Buildfile: build.xml does not exist!
Build failed

step 2: Execute test suites :: To generate report in TestNg format:

Now in terminal, navigate to the project directory which has your ‘build.xml’. Then run following command:

Yash@YASH-PC ~
$ cd git/sugarCRM/SugarCRM/
Yash@YASH-PC ~/git/sugarCRM/SugarCRM (master)
$ ant run

That’s it…This would run all test suites specified in build.xml

Note 1: ‘run’ in the above command is the ‘target'(ie, method/function in terms of java) name from ‘build.xml’ which executes the statements of those target.

Note 2: ‘run’ target depends on ‘compile’ target which in turn depends on ‘init’ & ‘clean’ target. Hence, the only command we need to run from terminal is ‘ant run’ from project directory, which in turn will also take care of init, clean, compile process.

(or) step 2: Execute test suites :: To generate report in XSLT format:

Instead of executing ‘run’ target, execute ‘runAsTestngXslt’ target. Done 🙂

Yash@YASH-PC ~
$ cd git/sugarCRM/SugarCRM/
Yash@YASH-PC ~/git/sugarCRM/SugarCRM (master)
$ ant runAsTestngXslt

 

Case 2: Multiple build.xml’s

Sometimes total number of test suites are too much to be executed all together (i.e, serial execution of test suites). Sometimes, based on our sub-products we wish to execute test suites separately & do not wish to configure single build.xml for all test suites.

In such case, we can opt for multiple build.xml files where each one is targeted to execute specific set of test suites based on sub-products or our specific context.

Hence, this results in multiple build.xml files out of which one has to be a parent build.xml which in turn can call our individual child build.xml one after another.

Note: We may directly execute child build.xml from terminal but generally it is a door towards issues which are pertaining to identifying appropriate directory path for all the variables declared within the file. As per the ‘Ant’ developers it is recommended that you have a build.xml in root of project directory. Therefore we can have one parent build.xml in root directory which in turn takes care of calling child build.xml from sub-directories.

Build.xml content:

In this case, parent build.xml has to be placed in the ‘root’ of project directory system. All Child build.xml can be stored under single folder like ‘masterBuildFiles’. We may also rename the child build.xml based on our context.

Content of Parent build.xml:

<?xml version="1.0" encoding="UTF-8"?>
<project name="novopayBoi" default="clean" basedir=".">

 <!-- cleaning the destination folders -->
 <target name="clean">
 <echo message="deleting existing build directory"/>
 <delete dir="${test.dest}"/>
 </target>

 <target name="run">
 <!-- include any number of child build.xml as you wish-->
 <ant antfile="masterBuildFiles/buildXYZ_Template.xml" target="run" />
 <ant antfile="masterBuildFiles/build1_WebSanity.xml" target="run" />
 <ant antfile="masterBuildFiles/build2_WebRegression.xml" target="run" />
 </target>

</project>

Content of each Child build.xml: (Note: for each child build.xml we need to configure appropriately which suites to be executed within target ‘run’)

Note: The only difference between this child build.xml & the build.xml from case 1 above is the reference to ‘basedir’:

<project name="projectName" default="clean" basedir="./../../"> <!-- 2 directories back from child build.xml's current directory which is pointing to root directory -->

Remaining part is the same as single build.xml code sample from case 1.

That’s all it takes to generate test automation suite reports in TestNg or XSLT format.

Happy Testing!!