Skip to content

Make sure debug assertions work in tests #3632

Description

@lann

We currently run a lot of tests with --release for performance reasons (wasmtime cranelift compile times):

spin/Makefile

Lines 57 to 76 in 291316d

# Run the runtime tests without the tests that use some sort of assumed external dependency (e.g., Docker, a language toolchain, etc.)
.PHONY: test-runtime
test-runtime:
cargo test --release runtime_tests --no-default-features --no-fail-fast -- --nocapture
# Run all of the runtime tests including those that use some sort of assumed external dependency (e.g., Docker, a language toolchain, etc.)
.PHONY: test-runtime-full
test-runtime-full:
cargo test --release runtime_tests --no-default-features --features extern-dependencies-tests --no-fail-fast -- --nocapture
# Run the integration tests without the tests that use some sort of assumed external dependency (e.g., Docker, a language toolchain, etc.)
.PHONY: test-integration
test-integration: test-runtime
cargo test --release integration_tests --no-default-features --no-fail-fast -- --nocapture
# Run all of the integration tests including those that use some sort of assumed external dependency (e.g., Docker, a language toolchain, etc.)
.PHONY: test-integration-full
test-integration-full: test-runtime-full
cargo test --release integration_tests --no-default-features --features extern-dependencies-tests --no-fail-fast -- --nocapture

One major downside of this is that it disables debug assertions, which we definitely would rather have enabled for testing.

@alexcrichton has suggested that we can probably mitigate the performance issues by selectively enabling optimizations just for cranelift, e.g.:

[profile.test.package.cranelift-codegen]
opt-level = 3

I would probably do the same for [profile.dev] as well, for the same reasons.

Another option would be to continue testing with --release and selectively enable debug-assertions when testing, but there are other reasons to avoid --release such as - ironically - making compilation of the tests themselves faster.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions