A framework is only useful if you can run it flexibly. Run all tests. Run only smoke tests. Run against staging. Run in CI. This lesson covers all the ways to execute your framework.
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="API Test Suite" parallel="classes" thread-count="3">
<!-- Smoke tests — quick sanity check -->
<test name="Smoke Tests">
<groups>
<run>
<include name="smoke"/>
</run>
</groups>
<classes>
<class name="com.company.api.tests.UserTests"/>
<class name="com.company.api.tests.AccountTests"/>
</classes>
</test>
<!-- Full regression -->
<test name="Regression Tests">
<classes>
<class name="com.company.api.tests.UserTests"/>
<class name="com.company.api.tests.AccountTests"/>
<class name="com.company.api.tests.UserDataDrivenTests"/>
</classes>
</test>
</suite># Run all tests (default: dev environment)
mvn test
# Run against staging
mvn test -Denv=staging
# Run against production (read-only tests)
mvn test -Denv=prod
# Run only smoke group
mvn test -Dgroups=smoke
# Run specific test class
mvn test -Dtest=UserTests
# Run specific test method
mvn test -Dtest=UserTests#testCreateUser
# Run with a different testng.xml
mvn test -DsuiteXmlFile=testng-smoke.xml
# Skip tests (for build only)
mvn package -DskipTests
# Run with Allure results
mvn test && allure serve target/allure-results/<profiles>
<profile>
<id>dev</id>
<activation>
<activeByDefault>true</activeByDefault>
</activation>
<properties>
<env>dev</env>
</properties>
</profile>
<profile>
<id>staging</id>
<properties>
<env>staging</env>
</properties>
</profile>
<profile>
<id>prod</id>
<properties>
<env>prod</env>
</properties>
</profile>
<profile>
<id>smoke</id>
<properties>
<env>staging</env>
<groups>smoke</groups>
</properties>
</profile>
</profiles>With profiles, running smoke tests against staging becomes: mvn test -Psmoke. One flag instead of two.
name: API Tests
on:
push:
branches: [main, develop]
pull_request:
branches: [main]
schedule:
- cron: '0 6 * * *' # Daily at 6 AM
jobs:
smoke-tests:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Set up Java 17
uses: actions/setup-java@v4
with:
distribution: temurin
java-version: 17
- name: Run smoke tests
run: mvn test -Psmoke
- name: Upload Allure results
if: always()
uses: actions/upload-artifact@v4
with:
name: allure-results
path: target/allure-results/
regression-tests:
runs-on: ubuntu-latest
needs: smoke-tests
steps:
- uses: actions/checkout@v4
- name: Set up Java 17
uses: actions/setup-java@v4
with:
distribution: temurin
java-version: 17
- name: Run full regression
run: mvn test -Denv=staging
- name: Generate Allure report
if: always()
run: |
curl -o allure.tgz -Ls https://github.com/allure-framework/allure2/releases/download/2.25.0/allure-2.25.0.tgz
tar -xzf allure.tgz
allure-2.25.0/bin/allure generate target/allure-results/ -o allure-report/
- name: Upload Allure report
if: always()
uses: actions/upload-artifact@v4
with:
name: allure-report
path: allure-report/Developer pushes code to main or develop
GitHub Actions triggers the workflow
Smoke tests run first (quick, 2-3 minutes)
If smoke passes, regression tests run against staging
Allure report is generated and uploaded as artifact
Team gets notified on Slack if tests fail
Daily scheduled run catches environment-level issues overnight
Run smoke tests on every PR. Run full regression nightly. Run against production with read-only tests (GET only) weekly. This layered approach balances speed with coverage.
Q: How do you run your API tests in CI/CD?
A: We use GitHub Actions with two jobs. Smoke tests run on every PR and push — they take 2-3 minutes and cover critical flows. If smoke passes, full regression runs against staging. We use Maven profiles — mvn test -Psmoke for quick runs, mvn test -Denv=staging for environment-specific runs. Allure results are uploaded as build artifacts. We also have a daily scheduled run at 6 AM to catch overnight environment issues. The testng.xml controls which classes and groups run. For running specific tests locally, we use -Dtest=UserTests#testCreateUser.
Key Point: Use testng.xml for test organization, Maven profiles for environment presets, and CI/CD for automated runs. Smoke on every PR. Regression nightly. Allure reports as build artifacts.