Listeners are hooks into the TestNG lifecycle. They let you run custom code when a test starts, passes, fails, or finishes. The two most important listeners in Selenium automation are ITestListener (for screenshots and logging) and IRetryAnalyzer (for retrying flaky tests).
Think of listeners like security cameras in a building. They do not interfere with normal operations, but they record everything. When something goes wrong (a test fails), you have the footage (screenshot) to investigate.
ITestListener gives you callback methods for every lifecycle event. The most valuable one is onTestFailure — where you take a screenshot.
import org.testng.ITestListener;
import org.testng.ITestResult;
import org.testng.ITestContext;
import org.openqa.selenium.OutputType;
import org.openqa.selenium.TakesScreenshot;
import org.openqa.selenium.WebDriver;
import java.io.File;
import java.nio.file.Files;
import java.nio.file.Paths;
import java.text.SimpleDateFormat;
import java.util.Date;
public class TestListener implements ITestListener {
@Override
public void onStart(ITestContext context) {
System.out.println("\n=== Suite Started: "
+ context.getName() + " ===");
}
@Override
public void onFinish(ITestContext context) {
System.out.println("\n=== Suite Finished: "
+ context.getName() + " ===");
System.out.println("Passed: " + context.getPassedTests().size());
System.out.println("Failed: " + context.getFailedTests().size());
System.out.println("Skipped: " + context.getSkippedTests().size());
}
@Override
public void onTestStart(ITestResult result) {
System.out.println("\nStarting: "
+ result.getMethod().getMethodName());
}
@Override
public void onTestSuccess(ITestResult result) {
System.out.println("PASSED: "
+ result.getMethod().getMethodName());
}
@Override
public void onTestFailure(ITestResult result) {
System.out.println("FAILED: "
+ result.getMethod().getMethodName());
System.out.println("Reason: "
+ result.getThrowable().getMessage());
// Take a screenshot on failure
takeScreenshot(result);
}
@Override
public void onTestSkipped(ITestResult result) {
System.out.println("SKIPPED: "
+ result.getMethod().getMethodName());
}
private void takeScreenshot(ITestResult result) {
try {
Object testInstance = result.getInstance();
WebDriver driver = ((BaseTest) testInstance).getDriver();
File screenshot = ((TakesScreenshot) driver)
.getScreenshotAs(OutputType.FILE);
String timestamp = new SimpleDateFormat("yyyyMMdd_HHmmss")
.format(new Date());
String fileName = result.getMethod().getMethodName()
+ "_" + timestamp + ".png";
Files.createDirectories(Paths.get("screenshots"));
Files.copy(screenshot.toPath(),
Paths.get("screenshots/" + fileName));
System.out.println("Screenshot saved: screenshots/"
+ fileName);
} catch (Exception e) {
System.out.println("Could not take screenshot: "
+ e.getMessage());
}
}
}Flaky tests are a reality in Selenium. A test might fail because of a slow network, a JavaScript animation that took longer than usual, or a popup that appeared unexpectedly. IRetryAnalyzer lets you automatically retry a failed test before marking it as failed.
import org.testng.IRetryAnalyzer;
import org.testng.ITestResult;
public class RetryAnalyzer implements IRetryAnalyzer {
private int retryCount = 0;
private static final int MAX_RETRIES = 2;
@Override
public boolean retry(ITestResult result) {
if (retryCount < MAX_RETRIES) {
retryCount++;
System.out.println("Retrying "
+ result.getMethod().getMethodName()
+ " — attempt " + (retryCount + 1) + " of "
+ (MAX_RETRIES + 1));
return true; // true = retry the test
}
return false; // false = give up, mark as failed
}
}
// Apply to a specific test:
@Test(retryAnalyzer = RetryAnalyzer.class)
public void testFlakyDashboard() {
// If this fails, it retries up to 2 more times
}Adding retryAnalyzer to every @Test annotation is tedious. Use IAnnotationTransformer to apply it globally:
import org.testng.IAnnotationTransformer;
import org.testng.annotations.ITestAnnotation;
import java.lang.reflect.Constructor;
import java.lang.reflect.Method;
public class RetryTransformer implements IAnnotationTransformer {
@Override
public void transform(ITestAnnotation annotation,
Class testClass,
Constructor testConstructor,
Method testMethod) {
annotation.setRetryAnalyzer(RetryAnalyzer.class);
}
}Register the transformer in testng.xml:
<suite name="Suite with Retry">
<listeners>
<listener class-name="com.autopractice.listeners.TestListener" />
<listener class-name="com.autopractice.listeners.RetryTransformer" />
</listeners>
<test name="All Tests">
<classes>
<class name="com.autopractice.tests.BankingLoginTest" />
</classes>
</test>
</suite>Set MAX_RETRIES to 1 or 2, not higher. If a test fails 3 times, it is not flaky — it is broken. High retry counts mask real bugs and slow down your suite. Also, investigate why a test is flaky instead of just retrying it forever.
Q: How do you take a screenshot on test failure?
A: I implement ITestListener and override onTestFailure(). In that method, I get the WebDriver from the test class instance using result.getInstance(), cast it to TakesScreenshot, call getScreenshotAs(OutputType.FILE), and save it to a screenshots directory with the test name and timestamp. I register this listener in testng.xml so it applies to all tests automatically. Every failed test gets a screenshot for debugging.
Q: How do you handle flaky tests?
A: Two levels: (1) I use IRetryAnalyzer to automatically retry failed tests up to 2 times. I apply it globally using IAnnotationTransformer so I do not have to add retryAnalyzer to every @Test. (2) I investigate the root cause — usually it is a missing wait, a timing issue, or a shared state problem. Retrying is a band-aid, not a cure. The goal is to make tests stable enough that they never need retries.
Key Point: ITestListener for screenshots on failure. IRetryAnalyzer for retrying flaky tests. Register listeners in testng.xml.