feat(convert): read Thermo .raw directly, opt-in (convert.thermo_native, build feature thermo)

.github/workflows/ci.yml

@@ -55,6 +55,12 @@ jobs:
       - name: Clippy
         run: cargo clippy --workspace --all-targets --locked -- -D warnings
 
+      # The opt-in native Thermo reader (`--features thermo`, docs/04_convert.md) is not in
+      # any release build, so lint it here or it can rot unnoticed. Downloads the .NET host
+      # library at build time; needs no .NET runtime.
+      - name: Clippy (thermo feature)
+        run: cargo clippy -p mumdia --all-targets --locked --features thermo -- -D warnings
+
       # Rustdoc, warnings as errors. Not previously run anywhere, and it was failing:
       # four bracketed prose expressions (`[i0,i1,i2]`, `[0,1]`) were parsed as
       # intra-doc links to items that do not exist, and three public doc comments
@@ -92,8 +98,21 @@ jobs:
       # is a compile-time proc macro whose 1.0.15 is its final release, so there is
       # nothing to bump to and a hard failure would only train people to ignore
       # this job. Deny the class that has a fix.
+      #
+      # Three advisories are ignored deliberately: RUSTSEC-2026-0367, -0368 and -0369
+      # in zip 6.0.0 (SUID/SGID bits kept, symlinks outside the destination, and Unix
+      # permissions applied through symlinks, all during extraction). zip is only a
+      # build dependency of nethost-sys 0.7.3, reached only under the opt-in `thermo`
+      # feature (mzdata -> thermorawfilereader -> netcorehost ^0.18 -> nethost-sys): its
+      # build script downloads Microsoft's nethost NuGet package from nuget.org over TLS
+      # and unpacks it. Nothing extracts an archive at run time, and release builds do
+      # not enable the feature. The fix is zip 9.0, and no release in the chain reaches
+      # it (nethost-sys 0.8.0 still requires zip ^8.3). Drop the ignores when
+      # thermorawfilereader moves to a netcorehost whose nethost-sys requires zip >= 9.
       - name: Audit
-        run: cargo audit --deny unsound --deny yanked
+        run: >-
+          cargo audit --deny unsound --deny yanked
+          --ignore RUSTSEC-2026-0367 --ignore RUSTSEC-2026-0368 --ignore RUSTSEC-2026-0369
 
       # The desktop application has its own lockfile, which the step above does not
       # read (docs/29, work package C). One advisory is ignored deliberately:

CHANGELOG.md

@@ -115,6 +115,14 @@ than a number. Both are recorded in every run's `manifest.json`.
 
 ### Added
 
+- **Native Thermo `.raw` reading, opt-in** (`convert.thermo_native`, in a build with the
+  `thermo` Cargo feature). A `.raw` is read directly through mzdata and Thermo's
+  RawFileReader on a .NET 8 runtime, with the vendor centroiding ThermoRawFileParser uses,
+  and no mzML is written. On a 15 min Astral file the spectra are the same as
+  ThermoRawFileParser 1.4.5's (every peak's m/z; intensities except 2 tied peaks of 250 M),
+  in 81 s against 787 s of conversion plus a 3.6 GB mzML; on a six-file HYE experiment
+  159,740 against 159,525 peptides at 1%. Release builds do not enable the feature
+  (docs/04_convert.md, "Native Thermo reading").
 - **`mbr.second_pass_prefilter_keep_decoys`: the rescore prefilter of the second pass alone.**
   The first-pass pool of a match-between-runs experiment holds every candidate of every run
   and needs the prefilter for memory; the second-pass pool holds the empirical library's

README.md

@@ -131,8 +131,9 @@ experiment the TSVs are experiment-wide, with one quantity column per run. Detai
 ## Limits
 
 - Input is mzML or a Bruker diaPASEF `.d`. Thermo `.raw` is converted with
-  ThermoRawFileParser; other vendor formats go through ProteoWizard `msconvert` and are
-  not verified end to end ([`docs/04_convert.md`](docs/04_convert.md)).
+  ThermoRawFileParser, or read directly by a build with the opt-in `thermo` feature; other
+  vendor formats go through ProteoWizard `msconvert` and are not verified end to end
+  ([`docs/04_convert.md`](docs/04_convert.md)).
 - Wildcard and terminal variable modifications are not supported.
 - CI builds and tests the engine on three platforms and runs an end-to-end smoke test, but
   does not run the Python sidecars on data. The PyTorch rescorer and the DeepLC fine-tune

configs/config-schema.json

@@ -53,6 +53,21 @@
       "gates": [],
       "source_struct": "ConvertConfig"
     },
+    {
+      "path": "convert.thermo_native",
+      "name": "thermo_native",
+      "section": "convert",
+      "rust_type": "bool",
+      "kind": "bool",
+      "optional": false,
+      "default": false,
+      "default_text": "false",
+      "help": "Read a Thermo `.raw` directly instead of converting it to mzML first: through mzdata and Thermo's RawFileReader, with the vendor centroiding ThermoRawFileParser also uses. No mzML is written. Needs an engine built with the `thermo` feature and a .NET 8 runtime on the machine, and is subject to Thermo's RawFileReader licence. Off by default (benchmark-gated, docs/04_convert.md, \"Native Thermo reading\").",
+      "gates": [
+        "benchmark-gated"
+      ],
+      "source_struct": "ConvertConfig"
+    },
     {
       "path": "convert.msconvert",
       "name": "msconvert",

docs/04_convert.md

@@ -550,7 +550,7 @@ vendor file was involved.
 |---|---|---|
 | mzML | none | supported |
 | Bruker timsTOF `.d` (dir) with diaPASEF frames | none: native reader (timsrust), ion mobility kept | supported; the 1/K0 calibration, gate, features, retrace and cross-run quant are opt-in (`configs/examples/diapasef.json`, docs/35_diapasef.md) |
-| Thermo `.raw` (file) | ThermoRawFileParser, or msconvert | **exercised end to end** (doxy, 2026-09-06: a 3.7 GB Astral `.raw` through `mumdia convert --mzml x.raw`, ThermoRawFileParser 2.0.0 found by `auto`, 6:40 and 4.4 GB for the converter, 3.4 GB mzML renamed into place beside the input, reused by the next run in 1.3 s; `peak-census` on the same `.raw` likewise) |
+| Thermo `.raw` (file) | ThermoRawFileParser, or msconvert; or none under `convert.thermo_native` in a build with the `thermo` feature (opt-in, below) | **exercised end to end** (doxy, 2026-09-06: a 3.7 GB Astral `.raw` through `mumdia convert --mzml x.raw`, ThermoRawFileParser 2.0.0 found by `auto`, 6:40 and 4.4 GB for the converter, 3.4 GB mzML renamed into place beside the input, reused by the next run in 1.3 s; `peak-census` on the same `.raw` likewise) |
 | Bruker `.d` (dir), any other | msconvert | wired, unverified; **ion mobility is discarded** |
 | SCIEX `.wiff` / `.wiff2` | msconvert | wired, unverified; a `.wiff` needs its `.wiff.scan` companion beside it, and the engine names that file when msconvert fails without it |
 | Agilent `.d` (dir) | msconvert | wired, unverified |
@@ -574,7 +574,9 @@ Thermo and SCIEX, a .NET runtime, while the workspace pins `mzdata` to
 `default-features = false, features = ["mzml", "miniz_oxide"]` so that building
 MuMDIA needs no vendor SDK, C++ or .NET toolchain. Linking a vendor reader imposes
 that on every build on every platform, including the ones that never see a vendor
-file.
+file. The Thermo reader is now available behind an opt-in build feature
+("Native Thermo reading" below), which keeps the default build exactly as free of
+those toolchains as before.
 
 The native diaPASEF reader is the exception, because it needs none of those: a TDF is
 an SQLite index plus zstd-compressed frames. timsrust 0.4.2 (with only its `tdf`
@@ -599,6 +601,49 @@ decode failure. `peak-census` on such a `.d` counts after the native centroiding
 docs/35_diapasef.md, section 2, has the reader in full, its settings, its measured
 motivation and its scope.
 
+### Native Thermo reading
+
+A build with the `thermo` Cargo feature (`cargo build --release --features thermo`) can
+read a Thermo `.raw` directly, through mzdata and Thermo's own RawFileReader hosted on a
+.NET 8 runtime, when `convert.thermo_native` is set. `raw::ensure_mzml` then passes the
+`.raw` through unconverted and `convert` reads it (`Source::Thermo`): no mzML is written
+and no ThermoRawFileParser is needed. The spectra are the vendor centroids, the same peak
+picking ThermoRawFileParser applies by default, so the two routes give the same spectra.
+One thread reads the file, which RawFileReader does serially, and the thread pool
+decodes the spectra in batches that are folded back in file order, byte-identical to a
+one-thread decode. The instrument comes from the file's metadata (the model is the first
+instrument-configuration parameter that is neither the serial number nor the instrument
+method text, which mzdata stores as `customization`), and the spectra reports record
+`params.reader = "thermorawfilereader"` and `params.centroiding = "vendor"`.
+
+Measured against ThermoRawFileParser 1.4.5 (`-f 2 -m 2`, peak picking on, the arguments
+`raw.rs` uses) on PXD071205 Astral files, 2026-10-19:
+
+| | native | via ThermoRawFileParser |
+|---|---|---|
+| one 15 min file, spectra | 2,022 MS1 / 188,011 MS2, 250 windows | identical counts, windows and MS2-to-MS1 map |
+| peaks | 249,976,110 MS2 | identical m/z everywhere; intensities identical except 2 tied peaks at one m/z stored in swapped order |
+| time and disk for that file | convert 81 s, no mzML | conversion 787 s (mono) and a 3.6 GB mzML, then convert 20 s |
+| six-file HYE experiment, peptides at 1% | 159,740 | 159,525 (NN seed noise) |
+
+Requirements and limits:
+
+- A .NET 8 runtime on the machine that runs the search (`DOTNET_ROOT` when it is not in the
+  default location). The build downloads the .NET host library (`nethost`): the build script
+  of `nethost-sys` 0.7.3 fetches Microsoft's NuGet package and unpacks it with zip 6.0.0,
+  which carries three extraction advisories (RUSTSEC-2026-0367, -0368, -0369). They concern
+  unpacking an untrusted archive, which happens only at build time and only for that
+  package, and no release in the chain reaches the fixed zip 9.0, so CI's `cargo audit`
+  ignores them until one does (`.github/workflows/ci.yml`).
+- Windows: the host library is built against the static C runtime, so the build needs
+  `RUSTFLAGS="-C target-feature=+crt-static"`. Linux: a glibc build; the fully static musl
+  release target cannot host the runtime.
+- Using RawFileReader means accepting Thermo's RawFileReader licence. The release
+  archives, the Docker image and the desktop application are built without the feature,
+  so nothing MuMDIA ships includes it.
+- Off by default and benchmark-gated: a build without the feature refuses
+  `convert.thermo_native` with a message rather than falling back to a converter.
+
 ### Two converters, and the licence difference between them
 
 ThermoRawFileParser is Apache-2.0 and from CompOmics, so the desktop application

docs/24_config_reference.md

@@ -52,7 +52,7 @@ undocumented on purpose; those fields are counted under "Coverage".
 | Section | Struct | Fields | Stage document |
 |---|---|---|---|
 | [(top level)](#top-level) | `Config` | 18 | [docs/02_config_and_data_model.md](02_config_and_data_model.md) |
-| [`convert`](#convert) | `ConvertConfig` | 13 |  |
+| [`convert`](#convert) | `ConvertConfig` | 14 |  |
 | [`prescan`](#prescan) | `PrescanConfig` | 7 | [docs/21_prescan.md](21_prescan.md) |
 | [`prescreen`](#prescreen) | `PrescreenConfig` | 31 |  |
 | [`prescreen.tags`](#prescreentags) | `PrescreenTagsConfig` | 7 |  |
@@ -110,6 +110,7 @@ Vendor-format conversion, read by every subcommand that takes a spectra path (`r
 | Field | Type | Default | Gated | Description |
 |---|---|---|---|---|
 | `thermo_raw_parser` | `String` | `"auto"` |  | Path to the ThermoRawFileParser executable, or `"auto"` to search. `"auto"` looks at `MUMDIA_THERMO_PARSER`, then beside the engine binary, then on `PATH`. Empty means the same as `"auto"`; a real path is used verbatim and its absence is an error rather than a silent fallback, because a fallback would convert with a different program than the one asked for and vendor conversion is not reproducible across converters. |
+| `thermo_native` | `bool` | `false` | benchmark-gated | Read a Thermo `.raw` directly instead of converting it to mzML first: through mzdata and Thermo's RawFileReader, with the vendor centroiding ThermoRawFileParser also uses. No mzML is written. Needs an engine built with the `thermo` feature and a .NET 8 runtime on the machine, and is subject to Thermo's RawFileReader licence. Off by default (benchmark-gated, docs/04_convert.md, "Native Thermo reading"). |
 | `msconvert` | `String` | `"auto"` |  | Path to ProteoWizard `msconvert`, or `"auto"` to search. Used for every vendor format except Thermo, which prefers ThermoRawFileParser: Bruker `.d`, SCIEX `.wiff`, Agilent `.d` and Waters `.raw`. It is also the Thermo fallback when no ThermoRawFileParser is found. `"auto"` searches `MUMDIA_MSCONVERT`, beside the engine binary, the version-stamped ProteoWizard directories under Program Files on Windows (newest first), then `PATH`. MuMDIA never ships or downloads ProteoWizard. Its vendor readers bundle the instrument vendors' own libraries under the vendors' licence terms, which the user accepts when obtaining it, and automating that acceptance is not MuMDIA's to do. |
 | `msconvert_args` | `Vec<String>` | `[]` |  | Extra arguments appended to every `msconvert` invocation. An escape hatch, not a tuning surface. The per-vendor defaults already request indexed 64-bit zlib mzML, vendor peak picking where it exists, and `--combineIonMobilitySpectra` for Bruker. Use this for something the defaults cannot express, such as an `--filter` that trims an acquisition. Arguments are passed through verbatim and are not validated. |
 | `reuse_converted` | `bool` | `true` |  | Reuse an mzML that already sits beside the `.raw` and is newer than it. On by default: conversion is minutes per file and its output is deterministic given the same converter, so re-running a search should not pay for it twice. Turn it off when the neighbouring mzML may have come from a different converter or a different `.raw` of the same name. |
@@ -1268,6 +1269,6 @@ instead of looking like an absent field.
 
 ## Coverage
 
-24 structs and 318 fields emitted from `rust/mumdia/crates/mumdia-core/src/config.rs`, plus 40 enumerations, 1 named profile(s), 101 environment variables read and 20 set.
+24 structs and 319 fields emitted from `rust/mumdia/crates/mumdia-core/src/config.rs`, plus 40 enumerations, 1 named profile(s), 101 environment variables read and 20 set.
 
-33 field(s) carry a gating marker in their doc comment. 59 field(s) carry no doc comment at all, so their description is empty above. 2 default(s) could not be resolved and 2 have none by design.
+34 field(s) carry a gating marker in their doc comment. 59 field(s) carry no doc comment at all, so their description is empty above. 2 default(s) could not be resolved and 2 have none by design.