How to run PHPUnit tests in a random order

Sometimes you can end up in a situation where your tests only pass if they are run in a specific order. This means that if you change one test, another test can fail. Not great.

Each test should start with a clean slate, it shouldn't depend on left-overs from previous tests. Tests that depend on each other can hard to notice, because you usually run your tests in the same order.

An effective way to detect this problem in your testsuite is by shuffling your tests. You can run your tests in a random order using this PHPUnit flag:

# For unit tests:
vendor/bin/phpunit --order-by=random

# For Laravel Dusk tests:
php artisan dusk --order-by=random

Shuffling parallel tests in CI

If you run your tests in CI in parallel, then you need to make sure each runner shuffles the tests in a deterministic way. You can achieve this by using the --random-order-seed flag. For example, when using GitHub Actions:

- name: Run PHPUnit tests
  if: startsWith(matrix.parallel-runner, 'phpunit')
  run: vendor/bin/phpunit --order-by=random --random-order-seed=$GITHUB_RUN_ID
  env:
    PARALLEL_TEST_RUNNER: ${{ matrix.parallel-runner }}

- name: Run Dusk Tests
  if: startsWith(matrix.parallel-runner, 'dusk')
  run: php artisan dusk --order-by=random --random-order-seed=$GITHUB_RUN_ID
  env:
    PARALLEL_TEST_RUNNER: ${{ matrix.parallel-runner }}

The $GITHUB_RUN_ID variable is an integer that is unique for each workflow run within a repository. If we use this variable as the seed, each workflow run will shuffle the tests in a different order, but each runner in the same workflow will shuffle the tests in the same order.