Skip to content

feat(convert): read Thermo .raw directly, opt-in (convert.thermo_native, build feature thermo) - #217

Merged
RobbinBouwmeester merged 8 commits into
mainfrom
feat/thermo-native
Oct 25, 2026
Merged

RobbinBouwmeester merged 8 commits into
mainfrom
feat/thermo-native

Conversation

@RobbinBouwmeester

Copy link
Copy Markdown
Member

Reads a Thermo .raw directly instead of converting it to mzML first. This is opt-in at two levels: a Cargo feature (thermo) that no release build enables, and a configuration key (convert.thermo_native, default false).

How it works

  • mzdata's thermo feature hosts Thermo's RawFileReader on a .NET 8 runtime.
  • raw::ensure_mzml passes the .raw through under convert.thermo_native. convert reads it with vendor centroiding, the same peak picking ThermoRawFileParser applies by default.
  • One thread reads the file, which RawFileReader does serially. The thread pool decodes the spectra in batches, folded back in file order; the output is byte-identical to a one-thread decode.
  • Spectra that arrive as a centroid list without signal arrays are handled in peaks_of. The mzML path is unchanged.
  • The instrument comes from the file's metadata. instrument::from_configurations skips the serial number and the instrument-method text that mzdata stores as customization, so the Astral defaults apply as they do for an mzML.
  • Reports record params.reader = "thermorawfilereader" and params.centroiding = "vendor".
  • A build without the feature refuses convert.thermo_native with a message rather than falling back to a converter.

Measured against ThermoRawFileParser 1.4.5 on PXD071205 Astral files (docs/04_convert.md, "Native Thermo reading"):

native via ThermoRawFileParser
one 15 min file 2,022 MS1 / 188,011 MS2 spectra, 250 windows identical counts, windows and MS2-to-MS1 map
peaks 249,976,110 MS2 every peak's m/z identical; intensities identical except 2 tied peaks at one m/z in swapped order
time and disk convert 81 s (before the parallel decode), no mzML conversion 787 s under mono plus a 3.6 GB mzML, then convert 20 s
six-file HYE experiment, peptides at 1% 159,740 159,525 (within NN seed noise)

The spectra of all six runs were compared one by one, with the same result. The parallel decode is byte-identical to the one-thread decode on a 5 min file. Its timing on the 15 min file will follow once the test host is free.

Requirements and limits

  • A .NET 8 runtime on the machine that runs the search (DOTNET_ROOT when it is not in the default location).
  • Windows builds need RUSTFLAGS="-C target-feature=+crt-static", because the downloaded .NET host library uses the static C runtime.
  • Linux needs a glibc build; the static musl target cannot host .NET.
  • Thermo's RawFileReader licence applies to whoever builds or uses it. Nothing MuMDIA ships enables the feature.

Checks

  • clippy, with and without the feature, is clean.
  • Workspace tests and rustdoc pass, and the config reference is regenerated.
  • CI gains a lint step for the feature, so it cannot rot unnoticed.

🤖 Generated with Claude Code

RobbinBouwmeester and others added 8 commits October 19, 2026 08:50
…mo_native, build feature thermo)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…OG and a CI lint

One thread reads the .raw through RawFileReader, which is serial, and the
thread pool decodes the spectra in batches of 512 that are folded back in file
order: byte-identical spectra artifacts to the one-thread decode.

docs/04_convert.md gains "Native Thermo reading" (mechanism, the measured
comparison with ThermoRawFileParser 1.4.5 on PXD071205 Astral files, the .NET 8
runtime, the Windows crt-static and Linux glibc build requirements, and the
RawFileReader licence; release builds do not enable the feature). CI lints the
`thermo` feature, which no release build enables, so it cannot rot unnoticed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Conflicts: CHANGELOG.md (the native Thermo entry goes under Added in main's
Unreleased section, after the 0.8.0 release) and the generated
docs/24_config_reference.md (regenerated).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…2026-0382, -0381)

Both reach the lockfile through the thermo feature (nethost-sys's build-time
NuGet download); semver-compatible updates. THIRD_PARTY_LICENSES.md and
sbom.cdx.json regenerated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CHANGELOG.md: both Unreleased Added entries kept (native Thermo reading,
mbr.second_pass_prefilter_keep_decoys); docs/24 and the config schema
regenerated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
docs/24_config_reference.md and configs/config-schema.json regenerated: the
GitHub-side merge 7ba0400 left them stale against main's new search_seed
keys, which failed ci/gen_config_reference.py --check.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e's build chain

RUSTSEC-2026-0367, -0368 and -0369 (zip extraction: SUID/SGID bits kept,
symlinks outside the destination, permissions applied through symlinks)
reach the lockfile only through the opt-in thermo feature: mzdata ->
thermorawfilereader 0.8.0 -> netcorehost ^0.18 -> nethost-sys 0.7.3, whose
build script downloads Microsoft's nethost NuGet package from nuget.org and
unpacks it with zip 6.0.0. Nothing extracts an archive at run time and
release builds do not enable the feature. The fix is zip 9.0, and no release
in the chain reaches it (nethost-sys 0.8.0 still requires zip ^8.3), so the
engine audit ignores the three IDs, with the reason and the removal
condition in the workflow comment and in docs/04 ("Native Thermo reading").

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@RobbinBouwmeester
RobbinBouwmeester merged commit a836630 into main Oct 25, 2026
12 checks passed
@RobbinBouwmeester RobbinBouwmeester mentioned this pull request Oct 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant