[ekamran]

Fixes #48.

Problem

When a test fails on a host for a reason the host already knows about, there is no simple way to leave it out. The only option is WPT_PHPUNIT_CMD, which replaces the whole PHPUnit command, so the host has to rebuild cd <test dir> && php ./vendor/phpunit/phpunit/phpunit --dont-report-useless-tests --verbose and the WPT_FLAVOR / WPT_EXTRATESTS flags by hand. None of this is documented.

Fix

• New optional WPT_PHPUNIT_ARGS. test.php adds it to the end of the default PHPUnit command, after the multisite and extra-group flags. It is empty by default, so the command does not change for existing setups.
• When WPT_PHPUNIT_CMD is set, WPT_PHPUNIT_ARGS is ignored and test.php logs that, because WPT_PHPUNIT_CMD replaces the whole command.
• compose.yaml passes WPT_PHPUNIT_ARGS to the runner container, next to WPT_PHPUNIT_CMD, because only the variables listed there reach it.
• .env.default and the README document it. The new README section "Skipping tests that fail for a known reason" covers:
  • one test: --filter with a negative lookahead pattern. It also leaves out every data set of that test and keeps tests whose names only start the same way.
  • a group: --exclude-group. PHPUnit then ignores the <exclude> list in phpunit.xml.dist (or tests/phpunit/multisite.xml), so groups such as ajax and external-http would run unless that list is added too. The README gives the single site and multisite lists separately, since they differ. With WPT_EXTRATESTS set, only the failing group should be listed.

Verification

test.php end to end, re-run after merging master (#361), with a stub vendor/phpunit/phpunit/phpunit that prints the arguments it receives and the .env sourced as the README describes:

• no WPT_PHPUNIT_ARGS: the same command as master.
• the README --filter example: arrives as one argument with the pattern intact.
• WPT_FLAVOR=1, WPT_EXTRATESTS=1 and --exclude-group: the arguments come after -c tests/phpunit/multisite.xml --group ajax.
• WPT_PHPUNIT_CMD set: the arguments are ignored and the message is logged.
• WPT_SSH_CONNECT set, with a stand-in ssh that runs the remote command in a shell: the pattern arrives intact.
• whitespace only: treated as empty.

Docker Compose: with the README --filter example set on the host, docker compose run on the runner image built from this branch received WPT_PHPUNIT_ARGS unchanged (checked with printenv; the test suite was not run in the container).

Verified with PHPUnit 9.6.38 on a small suite whose config excludes ajax:

• the --filter pattern leaves out exactly the named test and all of its data sets, and keeps a test whose name only starts the same way and a method with the same name in another class.
• --exclude-group with only the failing group makes the ajax test run. Adding the config's list keeps it out.
• --group ajax with an exclude list that contains ajax runs no tests.

php -l and PHPCS with the repository ruleset are clean on test.php.

Notes

• Add multi-PHP, environment labels, and commit tracking. #333 rewrites the same lines in test.php, and Add multi-PHP, environment labels, and commit tracking. #333 and Add GitHub action for easy remote test execution via GitHub Actions #245 already conflict with master. This branch merges cleanly with the other open PRs that touch the same files (Reduce the need for npm by installing built files from the WordPress mirror #246, Add host-tools/testrunner.sh to run the full test cycle in one step #346, Add host-tools/setup.sh to create .env with prompts #348, Document the host tools #349).

Use of AI

AI assistance: Yes
Tool(s): Claude Code and Codex
Used for: Investigation, implementation, verification harness, and PR wording. I reviewed the reasoning and test results, and I take responsibility for the contribution.

[github-actions]

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs.

If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message.

Co-authored-by: ekamran <ekamran@git.wordpress.org>
Co-authored-by: timiwahalahti <sippis@git.wordpress.org>
Co-authored-by: danielbachhuber <danielbachhuber@git.wordpress.org>

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

---

Props Bot commands

You can trigger Props Bot actions by commenting on this PR:

• @propsbot add <username> to give props to a GitHub user. Requires write access.
• @propsbot remove <username> to leave a user out of the props. Requires write access.
• @propsbot refresh to refresh this list. Adding the props-bot label works too.

[github-actions]

✅ Container Tests: success

Results for a36fbf1 from workflow run #19. Logs, junit.xml, and env.json are attached to the run as artifacts.

PHPUnit Test Runner: ✅ passed

• PHP: 8.1.34
• Database: mariadb:10.6

Step	Status	Exit code	Duration
prepare	✅ passed	0	58s
test	✅ passed	0	515s
report	✅ passed	0	0s
cleanup	✅ passed	0	17s

Test results

Tests	Failures	Errors	Skipped	Time
31161	0	0	68	377s

---

PHPUnit Test Runner: ✅ passed

• PHP: 8.3.36
• Database: mysql:8.4

Step	Status	Exit code	Duration
prepare	✅ passed	0	59s
test	✅ passed	0	496s
report	✅ passed	0	0s
cleanup	✅ passed	0	16s

Test results

Tests	Failures	Errors	Skipped	Time
31161	0	0	63	344s