A test suite without CI is a test suite that will be forgotten. Let us wire everything up so tests run automatically on every PR and nightly for the full regression.
name: PR Tests
on:
pull_request:
branches: [main]
jobs:
smoke:
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 20
cache: 'npm'
- run: npm ci
- run: npx playwright install --with-deps chromium
- name: Run smoke tests (Chromium only)
run: npx playwright test --grep @smoke --project=chromium
- uses: actions/upload-artifact@v4
if: ${{ !cancelled() }}
with:
name: smoke-report
path: playwright-report/
retention-days: 7name: Nightly Regression
on:
schedule:
- cron: '0 2 * * 1-5'
workflow_dispatch: {}
jobs:
regression:
runs-on: ubuntu-latest
timeout-minutes: 45
strategy:
fail-fast: false
matrix:
shard: [1/3, 2/3, 3/3]
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 20
cache: 'npm'
- run: npm ci
- run: npx playwright install --with-deps
- name: Run all tests (shard ${{ matrix.shard }})
run: npx playwright test --shard=${{ matrix.shard }}
- uses: actions/upload-artifact@v4
if: ${{ !cancelled() }}
with:
name: report-${{ strategy.job-index }}
path: playwright-report/
retention-days: 30| Aspect | PR Pipeline | Nightly Pipeline |
|---|---|---|
| Trigger | Every pull request | Cron schedule (2 AM weekdays) |
| Tests | @smoke tagged only | All tests |
| Browsers | Chromium only | Chromium + Firefox + WebKit |
| Sharding | No (fast enough) | Yes (3 shards) |
| Purpose | Fast feedback, block bad PRs | Catch cross-browser and regression issues |
| Time | 3-5 minutes | 15-20 minutes (with sharding) |
| Report retention | 7 days | 30 days |
Install only the browsers you need. For PR runs, npx playwright install --with-deps chromium installs just Chromium -- 3x faster than installing all browsers. For nightly runs, install everything with npx playwright install --with-deps.
Key Point: Use two CI pipelines: a fast PR pipeline (smoke tests, Chromium only) and a nightly regression pipeline (all tests, all browsers, sharded). This balances speed and coverage.
Key Point: Split CI into a fast PR pipeline for smoke tests and a comprehensive nightly pipeline for full regression across browsers