From a3aa89107d501c106c3edbd61431d17e6140ebce Mon Sep 17 00:00:00 2001 From: softglazee Date: Sat, 24 Oct 2026 06:15:14 +0500 Subject: [PATCH 1/2] List skipped and incomplete tests with their reasons in the test output PHPUnit lists failed and risky tests at the end of a run, but it shows skipped and incomplete tests only as a count unless it runs with --verbose. junit.xml marks those tests without a reason, so a host could not see anywhere why a test did not run. Run PHPUnit with --verbose, as WordPress core's own PHPUnit workflow does, and describe the lists in the README. The report sent to WordPress.org does not change. Fixes #47. --- README.md | 13 +++++++++++++ test.php | 13 +++++++++++-- 2 files changed, 24 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index c3659d6..6156c52 100644 --- a/README.md +++ b/README.md @@ -514,6 +514,19 @@ What do the symbols mean? `I` → Means that the test has been marked as incomplete. +`R` → Means that the test has been marked as risky, for example because it printed output. + +At the end of the run, PHPUnit lists the tests with errors, warnings, or failures, then the risky, incomplete, and skipped tests, each with the reason it gives. A skipped test often points to something the environment does not have, such as a PHP extension, so this list is worth reading even when no test fails. For example: + +``` +There were 62 skipped tests: +... +56) Tests_Sitemaps_wpSitemapsRenderer::test_get_sitemap_index_xml +This test requires the XSL extension. +``` + +`junit.xml` marks skipped and incomplete tests without the reason, so the reasons are only in this output. They are not part of the report sent to WordPress.org. + If you follow these steps, everything should work perfectly and not make any mistakes. In case you get any error, it may be normal due to some missing adjustment or extension of PHP, among others. We recommend that you adjust the configuration until it works correctly. After all, this tool is to help you improve the optimal configuration for WordPress in that infrastructure. ## Creating a report diff --git a/test.php b/test.php index 57a5d88..e2355f4 100644 --- a/test.php +++ b/test.php @@ -60,10 +60,19 @@ } unset( $wpt_extratests_ini ); -// Construct the command used for running the PHPUnit tests. +/* + * Construct the command used for running the PHPUnit tests. + * + * PHPUnit always lists the risky tests at the end of the run, but it only + * lists the skipped and incomplete tests, with the reason for each, when it + * runs with --verbose. junit.xml marks those tests without a reason, so this + * output is the only place where a host can see why a test did not run, for + * example because a PHP extension is missing. WordPress core's own PHPUnit + * workflow runs with --verbose too. + */ $wpt_phpunit_cmd = trim( getenv( 'WPT_PHPUNIT_CMD' ) ); if ( empty( $wpt_phpunit_cmd ) ) { - $wpt_phpunit_cmd = 'cd ' . escapeshellarg( $runner_vars['WPT_TEST_DIR'] ) . ' && ' . $runner_vars['WPT_PHP_EXECUTABLE'] . ' ./vendor/phpunit/phpunit/phpunit --dont-report-useless-tests' . $wpt_flavor_txt . $wpt_extratests_txt; + $wpt_phpunit_cmd = 'cd ' . escapeshellarg( $runner_vars['WPT_TEST_DIR'] ) . ' && ' . $runner_vars['WPT_PHP_EXECUTABLE'] . ' ./vendor/phpunit/phpunit/phpunit --dont-report-useless-tests --verbose' . $wpt_flavor_txt . $wpt_extratests_txt; } // If an SSH connection string is provided, prepend the SSH command to the PHPUnit execution command. From 94c9ac9971a1b7f92bb8570fddb1c8e1636ccece Mon Sep 17 00:00:00 2001 From: softglazee Date: Sun, 25 Oct 2026 00:51:37 +0500 Subject: [PATCH 2/2] Mention --verbose for a custom WPT_PHPUNIT_CMD in the README --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 6156c52..597ac7b 100644 --- a/README.md +++ b/README.md @@ -516,7 +516,7 @@ What do the symbols mean? `R` → Means that the test has been marked as risky, for example because it printed output. -At the end of the run, PHPUnit lists the tests with errors, warnings, or failures, then the risky, incomplete, and skipped tests, each with the reason it gives. A skipped test often points to something the environment does not have, such as a PHP extension, so this list is worth reading even when no test fails. For example: +At the end of the run, PHPUnit lists the tests with errors, warnings, or failures, then the risky, incomplete, and skipped tests, each with the reason it gives. The incomplete and skipped tests are listed because the runner's default PHPUnit command includes `--verbose`; add it to a custom `WPT_PHPUNIT_CMD` to see them there too. A skipped test often points to something the environment does not have, such as a PHP extension, so this list is worth reading even when no test fails. For example: ``` There were 62 skipped tests: