[iarif4u]

Fixes #260. Fixes #259 (creates the host-tools/ directory; @aialvi agreed to the handover in #260).

Purpose

Hosts say that running the runner is tedious (#257). This adds a ready-made script that runs the full cycle in one step, after the runner is set up.

What host-tools/testrunner.sh does

1. Goes to the runner directory: the parent of host-tools/, or WPT_RUNNER_DIR.
2. Stops with a clear message if .env does not exist.
3. Stops if another run is still active (flock), so cron runs cannot overlap.
4. Runs git pull --ff-only origin master. If the update fails, it warns and continues. Set WPT_SKIP_UPDATE=1 to skip the update.
5. Loads .env and runs prepare.php, test.php, report.php and cleanup.php with WPT_PHP_EXECUTABLE, the same PHP that the runner uses.

Differences from the example in the issue and the README:

• The report runs when tests fail. test.php exits with 1 when any test fails. A plain set -e script would then skip the report, so failures would never be reported.
• Cleanup always runs: after a failed prepare, a failed test, or a stopped run (trap).
• Exit status is 1 when any step fails, so cron mail or monitoring can see it.

The README "Script en Bash" section now points to this script. The systemd example is not changed.

Testing

I used stub prepare/test/report/cleanup.php files that exit 0 or 1:

Case	Result
No .env	Error, exit 1
All steps pass	All 4 run, exit 0
test.php fails	Report and cleanup still run, exit 1
prepare.php fails	Test and report skipped, cleanup runs, exit 1
cleanup.php fails	Exit 1
Second run while first is active	"Another test run is still active", exit 1
SIGTERM during test	Cleanup runs
git pull fails	Warning, run continues

Contributed at WordCamp Contributor Day.

[github-actions]

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs.

Unlinked Accounts

The following contributors have not linked their GitHub and WordPress.org accounts: @iarif4u, @jazzsequence.

Contributors, please read how to link your accounts to ensure your work is properly credited in WordPress releases.

If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message.

Unlinked contributors: iarif4u, jazzsequence.

Co-authored-by: kittenkamala <amykamala@git.wordpress.org>
Co-authored-by: ekamran <ekamran@git.wordpress.org>
Co-authored-by: mindctrl <mindctrl@git.wordpress.org>
Co-authored-by: ramonfincken <ramon-fincken@git.wordpress.org>
Co-authored-by: aialvi <aialvi@git.wordpress.org>

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

---

Props Bot commands

You can trigger Props Bot actions by commenting on this PR:

• @propsbot add <username> to give props to a GitHub user. Requires write access.
• @propsbot remove <username> to leave a user out of the props. Requires write access.
• @propsbot refresh to refresh this list. Adding the props-bot label works too.

[ramonfincken]

!!!!
Please refrain from using our hardcode paths @iarif4u

[iarif4u]

Thanks @ramonfincken, fair point. I took /home/wptestrunner from the existing README and the #260 example, but new scripts should not suggest one host's layout. Fixed in 41575c3: the examples now use /path/to/phpunit-test-runner, and the comment notes that the script finds the runner from its own location. I made the same change in #349 (387f77b).

[ekamran]

Thanks @iarif4u, this makes the runner much easier to schedule. I re-ran all eight cases from your table with stub steps, and they match. Shellcheck is clean.

Two cases break, details inline: SSH mode, and a second runner checkout on the same host. Both suggestions are tested; with them, only those two cases change and everything else behaves the same. Happy to approve once they are in.

[iarif4u]

@ekamran Both changes are in (28be5d0), plus the cron PATH line. I also wrapped source .env in set +u / set -u, the same fix as on #348. The test details are in my comment on #348. Could you take another look?

[ekamran]

Thanks @iarif4u. I re-checked 28be5d0 as a whole on macOS (bash 3.2) and Linux (bash 5.2, real flock): SSH mode runs the steps with the local php, a second checkout runs while the first is active, the same checkout is refused, a SIGTERM still runs the cleanup, and a .env with an unset variable no longer stops the script.

[iarif4u]

Thanks @ekamran for the approval, and for testing it on both macOS and Linux!

[ekamran]

Thanks @iarif4u. I re-checked 9edcc08 as a whole. The new mkdir claim is the same change I tested on October 28, byte for byte, and the PR still merges cleanly with master, including setup.sh from #348.

I ran it again on Linux (bash 5.2, real flock, MySQL 8.4) with the real prepare.php and cleanup.php. A second checkout, or a run started by hand with php prepare.php, now stops with the message, and the first run keeps its checkout and tables and passes. When two runs start at the same moment, only one claims the directory (3 of 3 tries). A leftover directory stops the next run until cleanup.php removes it.

On macOS (bash 3.2) with stub steps, a wrong PHP path no longer leaves the directory behind, and the signal and WPT_PHP_EXECUTABLE cases behave as before. A git clone into the empty claimed directory works, so prepare.php is unaffected.

One small follow-up, not blocking: the README still says the script "always cleans up". After a killed run, the next run now stops until cleanup.php runs, which the error message explains.

[iarif4u]

Thanks for the thorough re-check, ekamran! Took your README follow-up too — pushed 45ab9ff, it no longer says the script always cleans up.