Running tests is half the job. The other half is reports. If you run 500 tests at 2 AM and 12 fail, who is going to read the console log? Nobody. You need reports — visual, clickable, shareable. Jenkins supports two report plugins that every QA engineer should know: TestNG and Allure.
The TestNG Results Plugin reads the testng-results.xml file that TestNG generates automatically. It displays pass/fail counts, test duration, and error messages directly in the Jenkins build page. No extra code needed — just install the plugin and add one line to your Jenkinsfile.
post {
always {
// Parse and display TestNG results
testNG(
reportFilenamePattern: '**/testng-results.xml',
escapeTestDescription: true,
failureOnFailedTestConfig: false
)
}
}After adding this, every build page shows a "TestNG Results" link. Click it to see which tests passed, which failed, and the error messages. You also get a trend graph showing pass/fail counts over the last 20 builds. Very useful for spotting flaky tests.
Allure is the report that impresses managers and interviewers. Beautiful HTML reports with graphs, timelines, test history, categories, and screenshot attachments. If TestNG report is a text message, Allure is a full presentation deck.
stage('Publish Reports') {
steps {
// Generate Allure HTML report
allure([
results: [[path: 'target/allure-results']]
])
}
}
post {
always {
// TestNG results
testNG(reportFilenamePattern: '**/testng-results.xml')
// Archive raw reports for download
archiveArtifacts(
artifacts: 'target/surefire-reports/**,target/screenshots/**',
allowEmptyArchive: true
)
}
}<!-- pom.xml — Allure TestNG integration -->
<dependencies>
<dependency>
<groupId>io.qameta.allure</groupId>
<artifactId>allure-testng</artifactId>
<version>2.25.0</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>3.2.5</version>
<configuration>
<suiteXmlFiles>
<suiteXmlFile>${suiteXmlFile}</suiteXmlFile>
</suiteXmlFiles>
<argLine>
-javaagent:"${settings.localRepository}/org/aspectj/aspectjweaver/1.9.21/aspectjweaver-1.9.21.jar"
</argLine>
<systemPropertyVariables>
<allure.results.directory>target/allure-results</allure.results.directory>
</systemPropertyVariables>
</configuration>
</plugin>
</plugins>
</build>| Tab | What You See | Who Cares |
|---|---|---|
| Overview | Pass/fail pie chart, severity breakdown | Managers, leads |
| Suites | Test results grouped by class | QA engineers |
| Graphs | Trends, duration over builds | QA lead for flaky detection |
| Timeline | Parallel execution visualization | DevOps for performance tuning |
| Categories | Failures grouped by error type | QA for root cause analysis |
| Packages | Results grouped by package structure | Developers |
Attach screenshots to Allure on test failure. In your listener, use Allure.addAttachment("Screenshot", "image/png", screenshotBytes, ".png"). Now every failed test in the Allure report has a screenshot. This is interview gold.
Q: What test reports do you generate in your CI pipeline?
A: We generate two types of reports. TestNG Results are parsed by the Jenkins TestNG plugin — it shows pass/fail counts and trend graphs directly in the build page. Allure Reports are our primary reporting tool — they give us a rich HTML dashboard with pie charts, test history, failure categories, timelines, and attached screenshots. The Allure report is auto-generated in the pipeline and accessible via a link on the build page. We also archive surefire-reports and screenshots as Jenkins artifacts for download. On failure, the email notification includes a direct link to the Allure report so the team can investigate immediately.
Key Point: Use TestNG plugin for basic pass/fail tracking. Use Allure for rich visual reports with screenshots, trends, and categories.