Out of the box, Allure categorizes failures as "Product Defects" and "Test Defects" based on whether the failure was an assertion error or an exception. But you can create custom categories to match your team's workflow. You can also add environment information so the report shows which browser, OS, and environment the tests ran on.
Create a categories.json file in the allure-results directory. Allure reads this file and uses it to categorize failures. You define patterns that match error messages.
[
{
"name": "Element Not Found",
"description": "A locator could not find the element on the page. Likely a locator change or page not loaded.",
"matchedStatuses": ["broken"],
"messageRegex": ".*no such element.*|.*NoSuchElementException.*|.*Unable to locate element.*"
},
{
"name": "Timeout Failures",
"description": "A wait timed out. The page or element was too slow to load.",
"matchedStatuses": ["broken"],
"messageRegex": ".*TimeoutException.*|.*timed out.*|.*Expected condition failed.*"
},
{
"name": "Assertion Failures",
"description": "The test assertion failed. This is likely a real application bug.",
"matchedStatuses": ["failed"],
"messageRegex": ".*AssertionError.*|.*expected.*but found.*"
},
{
"name": "Stale Element Failures",
"description": "The element became stale — page refreshed or DOM changed during interaction.",
"matchedStatuses": ["broken"],
"messageRegex": ".*StaleElementReferenceException.*"
},
{
"name": "Infrastructure Issues",
"description": "Browser crashed, WebDriver session lost, or network error.",
"matchedStatuses": ["broken"],
"messageRegex": ".*SessionNotCreatedException.*|.*WebDriverException.*|.*connection refused.*"
}
]Copy this file to target/allure-results/ before generating the report. The easiest way is to use the Maven Resources plugin to copy it automatically.
<!-- Add to pom.xml build > plugins -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-resources-plugin</artifactId>
<version>3.3.1</version>
<executions>
<execution>
<id>copy-allure-categories</id>
<phase>test</phase>
<goals>
<goal>copy-resources</goal>
</goals>
<configuration>
<outputDirectory>
${project.build.directory}/allure-results
</outputDirectory>
<resources>
<resource>
<directory>src/test/resources/allure</directory>
<includes>
<include>categories.json</include>
<include>environment.properties</include>
</includes>
</resource>
</resources>
</configuration>
</execution>
</executions>
</plugin>This file adds context to the report overview. When someone opens the report, they immediately see which environment, browser, and OS the tests ran on.
# src/test/resources/allure/environment.properties
Browser=Chrome 125
Browser.Version=125.0.6422.78
OS=macOS Sonoma 14.5
Environment=QA
Base.URL=https://qa.autopractice.com
Java.Version=17.0.11
Selenium.Version=4.21.0
TestNG.Version=7.10.2For dynamic environment info (like the actual browser version used in the run), generate the environment.properties file programmatically in a @BeforeSuite method. Read the browser version from capabilities and write it to the file. This way the report always shows the actual environment, not a hardcoded value.
import java.io.FileWriter;
import java.util.Properties;
import org.openqa.selenium.Capabilities;
import org.openqa.selenium.remote.RemoteWebDriver;
import org.testng.annotations.BeforeSuite;
public class BaseTest {
@BeforeSuite
public void writeEnvironmentProperties() {
try {
Properties props = new Properties();
props.setProperty("Environment",
System.getProperty("env", "local"));
props.setProperty("Base.URL",
System.getProperty("baseUrl",
"https://autopractice.com"));
props.setProperty("Java.Version",
System.getProperty("java.version"));
props.setProperty("OS",
System.getProperty("os.name") + " "
+ System.getProperty("os.version"));
FileWriter writer = new FileWriter(
"target/allure-results/environment.properties");
props.store(writer, "Allure Environment");
writer.close();
} catch (Exception e) {
System.err.println("Could not write environment "
+ "properties: " + e.getMessage());
}
}
}Q: How do you categorize failures in your test reports?
A: I use a custom categories.json file in Allure that categorizes failures by their error type. Assertion failures are categorized as "Product Defects" — these are real application bugs. NoSuchElementException and TimeoutException are "Test Infrastructure Issues" — usually locator changes or slow page loads. StaleElementReferenceException gets its own category since it usually indicates a timing issue in the test. This categorization helps during triage meetings — we can quickly separate real bugs from test maintenance tasks.
Key Point: Custom categories turn a list of random failures into organized, actionable groups. Environment properties add context so everyone knows what was tested and where.