Skip to content

fix: name Swift Testing tests under their whole suite path - #42

Merged
esnunes merged 4 commits into
mainfrom
nested-suite-full-path
Nov 13, 2026
Merged

esnunes merged 4 commits into
mainfrom
nested-suite-full-path

Conversation

@esnunes

@esnunes esnunes commented Nov 11, 2026 •

Copy link
Copy Markdown
Member

What changed

  • Trait (CoverageAttributionTrait): the suite is now every suite the test is nested in, outermost first and joined with / (test.id.nameComponents.dropLast()). Before, it was only the innermost suite (components[count - 2]). nameComponents never includes the module (that's test.id.moduleName), so a top-level suite is Suite, a struct nested in a struct is Outer/Inner, a suite declared in extension Outer is Outer/Inner, and a suite in an enum namespace is Namespace/Inner. That one value feeds records.bin, unskippable.tsv and the skip-list lookup.
  • Skip list (load_skip_list()): the first component of a line is the module, the last is the test, and everything in between is the suite (Module/Outer/Inner/name()). test_coverage_attribution_is_skipped matches the full path first, then the innermost suite alone (Module/Inner/name()), which is what Tuist's server writes today. A line is ignored if it has fewer than three components or an empty one. Before, any line without exactly three components was dropped. XCTest is unchanged, since class names can't contain /.
  • README: the skip-list naming rules and the records.bin suite field.

Why

The result bundle and xcodebuild's -only-testing/-skip-testing name a nested Swift Testing test by its whole path (Module/Outer/Inner/b()). Tuist used to keep only the innermost suite. It now records the full Outer/Inner path: it migrated its server-side test-case IDs because quarantined tests in nested suites kept running. Because this package still recorded Inner:

  • Per-test coverage evidence for tests in nested suites no longer joined to Tuist's tests, so coverage-based selection always ran them. That loses savings but never gives wrong results.
  • The skip file couldn't express a nested test at all.

Compatibility with today's server

The server still names nested tests by their innermost suite until tuist/tuist#15645 deploys.

  • Skip list: works with both forms. A Module/Inner/name() line still skips the nested test, as it does today, and a Module/Outer/Inner/name() line skips it once the server sends full paths. The fallback keeps one ambiguity that already exists today: a line for a top-level Inner suite also matches a same-named test in a nested Outer.Inner.
  • Evidence: not compatible. A record holds one suite string, now Outer/Inner. The current server looks up evidence under Inner, so it finds none for nested-suite tests and runs them (:no_evidence) until #15645 deploys. Nothing is wrongly skipped; nested tests lose coverage-based savings in the meantime. Top-level suites and XCTest are unaffected.

Tests

  • New fixture Fixtures/Concurrency/Tests/ConcurrencyTests/SuitePathTests.swift covers a top-level suite, a struct nested in a struct, a suite nested ten levels deep, a suite in an extension, and a suite in an enum namespace.
  • recordsSwiftTestingTestsUnderTheirWholeSuitePath: checks that the records carry the full paths.
  • skipsTheListedSwiftTestingTestsOfNestedSuites: full-path lines and innermost-suite lines both skip nested tests (Swift 6.3+), while a path with the wrong outer suite matches nothing. The listed flag in unskippable.tsv asserts the matching on every Swift version, including those before 6.3, where nothing is cancelled.
  • Updated the expectations for the existing nested suites (MultiplicationTests/UnskippableMultiplicationTests and UnskippableSwiftTestingTests/SkippableTests) to use their full paths.

Validation

  • swift test (whole package, including the simulator UI and skip-list tests) on Xcode 26.3 / Swift 6.2.4: 14 tests passed.
  • swift test --filter 'skipsTheListed|runsEveryTestWithoutASkipList|WholeSuitePath|OfNestedSuites' on Xcode 26.6 / Swift 6.3.3, which compiles the Test.cancel path: passed.
  • After adding the innermost-suite fallback: the nested-suite and Swift Testing skip tests passed on both Xcode 26.3 and 26.6.
  • After adding the ten-level suite: the nested-suite and Swift Testing skip tests passed on both Xcode 26.3 and 26.6.
  • mise run lint: clean.

🤖 Generated with Claude Code

A test in a nested suite was recorded, reported and matched against the skip
list under its innermost suite alone (`Inner` for `Outer/Inner/b()`). The
result bundle and xcodebuild's -only-testing/-skip-testing name it by the whole
path, which Tuist now records too, so its evidence didn't join to the test and
the skip list couldn't name it.

The trait now names every suite the test is nested in, outermost first and
separated by `/`, and the skip list reads a line's first component as the
module, its last as the test and everything between as the suite. XCTest
names are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@esnunes
esnunes requested a review from a team as a code owner November 11, 2026 20:01
@esnunes
esnunes requested review from fortmarek and removed request for a team November 11, 2026 20:01
The skip list now joins everything between the module and the test into
the suite, and the trait looks tests up by the same joined identifier, so
`Module/Suite/a/b` matches a test whose display name is `a/b`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
esnunes and others added 2 commits November 13, 2026 11:18
…suite

Until Tuist's server records nested suites whole, it names a nested test
by its innermost suite (`Module/Inner/name()`), so a skip list it writes
would stop matching once the trait looks tests up by their full path.
test_coverage_attribution_is_skipped() now also tries the innermost
suite, so the same release works with skip lists of either form.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Records it under its whole path and skips it whether the list names that
path or only its innermost suite, so the path handling has no fixed depth.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@esnunes
esnunes merged commit 20da98b into main Nov 13, 2026
11 checks passed
@esnunes
esnunes deleted the nested-suite-full-path branch November 13, 2026 17:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant