Chapter 5: Build a Real Test Suite
Listeners are TestNG hooks that let you run custom code when specific events happen — test starts, test passes, test fails, test skips. Think of them like CCTV cameras watching your tests. They observe and react but do not interfere with the test logic itself.
ITestListener has methods for every test lifecycle event. You implement the ones you care about. The most useful: onTestFailure for capturing screenshots, onTestStart for logging, and onTestSuccess for recording timing.
package com.testerrank.listeners;
import com.testerrank.base.BaseTest;
import com.testerrank.utils.ScreenshotUtils;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.openqa.selenium.WebDriver;
import org.testng.ITestContext;
import org.testng.ITestListener;
import org.testng.ITestResult;
public class TestListener implements ITestListener {
private static final Logger logger =
LogManager.getLogger(TestListener.class);
@Override
public void onTestStart(ITestResult result) {
logger.info("========================================");
logger.info("STARTING: " + getTestName(result));
logger.info("========================================");
}
@Override
public void onTestSuccess(ITestResult result) {
long duration = result.getEndMillis()
- result.getStartMillis();
logger.info("PASSED: " + getTestName(result)
+ " (" + duration + "ms)");
}
@Override
public void onTestFailure(ITestResult result) {
logger.error("FAILED: " + getTestName(result));
logger.error("Reason: "
+ result.getThrowable().getMessage());
// Capture screenshot on failure
Object testInstance = result.getInstance();
if (testInstance instanceof BaseTest) {
WebDriver driver =
((BaseTest) testInstance).getDriver();
if (driver != null) {
String path = ScreenshotUtils.capture(
driver, result.getName());
logger.info("Screenshot: " + path);
}
}
}
@Override
public void onTestSkipped(ITestResult result) {
logger.warn("SKIPPED: " + getTestName(result));
if (result.getThrowable() != null) {
logger.warn("Skip reason: "
+ result.getThrowable().getMessage());
}
}
@Override
public void onStart(ITestContext context) {
logger.info("===== Suite Starting: "
+ context.getName() + " =====");
}
@Override
public void onFinish(ITestContext context) {
int passed = context.getPassedTests()
.getAllResults().size();
int failed = context.getFailedTests()
.getAllResults().size();
int skipped = context.getSkippedTests()
.getAllResults().size();
logger.info("===== Suite Finished: "
+ context.getName() + " =====");
logger.info("Results: " + passed + " passed, "
+ failed + " failed, " + skipped + " skipped");
}
private String getTestName(ITestResult result) {
return result.getTestClass().getRealClass().getSimpleName()
+ "." + result.getName();
}
}package com.testerrank.utils;
import org.openqa.selenium.OutputType;
import org.openqa.selenium.TakesScreenshot;
import org.openqa.selenium.WebDriver;
import java.io.File;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.time.LocalDateTime;
import java.time.format.DateTimeFormatter;
public class ScreenshotUtils {
private static final String DIR =
ConfigReader.get("screenshot.dir", "screenshots");
public static String capture(WebDriver driver,
String testName) {
try {
Path dir = Paths.get(DIR);
if (!Files.exists(dir)) {
Files.createDirectories(dir);
}
String timestamp = LocalDateTime.now().format(
DateTimeFormatter.ofPattern(
"yyyy-MM-dd_HH-mm-ss"));
String fileName = testName + "_"
+ timestamp + ".png";
Path filePath = dir.resolve(fileName);
File srcFile = ((TakesScreenshot) driver)
.getScreenshotAs(OutputType.FILE);
Files.copy(srcFile.toPath(), filePath);
return filePath.toString();
} catch (IOException e) {
System.err.println(
"Screenshot failed: " + e.getMessage());
return "";
}
}
// Useful for Allure report attachments
public static byte[] captureAsBytes(WebDriver driver) {
return ((TakesScreenshot) driver)
.getScreenshotAs(OutputType.BYTES);
}
}Some tests fail intermittently — network timeout, slow server, animation not completing. RetryAnalyzer automatically retries a failed test before marking it as a real failure. Use this carefully. Retrying hides real bugs if overused.
package com.testerrank.listeners;
import com.testerrank.utils.ConfigReader;
import org.testng.IRetryAnalyzer;
import org.testng.ITestResult;
public class RetryAnalyzer implements IRetryAnalyzer {
private int retryCount = 0;
@Override
public boolean retry(ITestResult result) {
int maxRetry = ConfigReader.getInt("retry.count", 1);
if (retryCount < maxRetry) {
retryCount++;
System.out.println("Retrying test: "
+ result.getName()
+ " (attempt " + retryCount
+ "/" + maxRetry + ")");
return true; // Yes, retry this test
}
return false; // No more retries
}
}
// Usage in test class:
// @Test(retryAnalyzer = RetryAnalyzer.class)
// public void testSomething() { ... }Q: What are TestNG listeners? Which ones do you use and why?
A: TestNG listeners are interfaces that let you hook into the test execution lifecycle. I primarily use two. ITestListener — I implement onTestFailure to capture screenshots automatically when any test fails, onTestStart/onTestSuccess for structured logging, and onFinish to print a summary of passed/failed/skipped counts. IRetryAnalyzer — I implement the retry method to automatically retry failed tests up to a configurable limit (usually 1) to handle intermittent failures like network timeouts. The listener is registered globally in testng.xml so it applies to all tests without adding annotations to each method.
Q: How do you capture screenshots on test failure automatically?
A: I use a TestNG ITestListener. In the onTestFailure callback, I get the test class instance from ITestResult, cast it to BaseTest to access the WebDriver, then use TakesScreenshot interface to capture the screenshot as a File. The screenshot is saved with a timestamped filename to the screenshots directory. The listener is registered in testng.xml, so every test class gets automatic screenshot capture without any extra code.
RetryAnalyzer should be set to 1 retry maximum in most cases. If a test needs more than 1 retry to pass consistently, the test itself is unstable and needs fixing — not more retries. Do not use retry as a band-aid for poorly written tests.
Notice that getDriver() in BaseTest must be protected (not private) so that TestListener can access it. This is a design consideration freshers miss — your listener and your base test need to work together.
Key Point: Listeners separate cross-cutting concerns (logging, screenshots, retries) from test logic. Register them in testng.xml for global effect.