Skip to content

🐛 sphinx-codelinks: read a compile-database entry's -- as the compiler does - #2510

Merged
chrisjsewell merged 6 commits into
masterfrom
claude/lucid-hawking-pa89x2-dashdash
Nov 13, 2026
Merged

chrisjsewell merged 6 commits into
masterfrom
claude/lucid-hawking-pa89x2-dashdash

Conversation

@chrisjsewell

@chrisjsewell chrisjsewell commented Nov 12, 2026 •

Copy link
Copy Markdown
Member

What

packages/sphinx-codelinks: the libclang engine now reads a compile_commands.json entry whose arguments contain -- correctly.

  • An options-ending -- (clang-cl ... -c -- src/a.cpp, as CMake writes it for clang-cl) is left out together with every argument after it (compile_db.filter_args), because those arguments are inputs.

  • A -- that is an option's value ends nothing, and the options after it are still read. That covers -Xlinker --, -mllvm --, -Xopenmp-target=<triple> --, -I -- and similar.

  • No -- is ever passed on to libclang. libclang reads the first literal -- in its arguments as the end of the options, whatever comes before it: it inserts -fsyntax-only ahead of that -- before the driver runs. So:

    • a pass-through option whose value is -- is left out together with that value;
    • -I, -include, -isystem, -iquote, -idirafter or -isysroot with a -- value is joined into one argument (-I--), which the driver reads the same way.
  • Leaving out a pass-through pair loses nothing a working entry has. Real clang rejects a -- handed to the compiler or the driver:

    • -Xclang -- and -Xpreprocessor -- give "unknown argument: '--'";
    • -Xarch_host -- gives "invalid Xarch argument".

    The other pass-through options hand the -- to a tool or stage that plays no part in preprocessing (the linker, the assembler, LLVM's backend). Nothing is promised for any other option whose value is -- (-x --, -D --).

  • The value of -Xarch_<arch> is one of the driver's own options, so it goes through the same rules as a bare option. -Xarch_host -MD, -c, -o out.o, the input and so on are left out together with their -Xarch_*. Passed on, -Xarch_host -MD would make the parse write a dependency file. The values of the other pass-through options are kept verbatim with them.

  • One list of pass-through options. filter_args asks builtin_headers.passes_next_argument, the predicate the -resource-dir placement already used. -mllvm is added to that list.

  • env_version is now 7, so an environment holding records extracted the old way is read again once.

Why

When the -- was passed on, the arguments libclang appends after the entry's own were read as input files. Those include the request for the detailed preprocessing record. As a result, the parse had no record and no skipped ranges, so the markers of every #if arm were extracted. An entry with -Xlinker -- could not be loaded at all, and all of its file's markers were dropped.

Tests

  • Two shared extraction cases in tests/data/extraction/preproc_compile_db.yaml, committed first and failing on the previous code:
    • options_terminator_in_entry_libclang: only the active arms' markers are extracted;
    • double_dash_value_of_pass_through_option_libclang (-Xlinker -- -DFEATURE): the #ifdef FEATURE marker is extracted and the #ifndef one is not.
  • tests/test_preproc_compile_db.py covers:
    • the terminator and the inputs after it, -- as the last argument, and two --;
    • every pass-through option followed by a -- and by another value;
    • the joined -I-- family and -o --;
    • -Xopenmp-target=<triple> -- against -Xopenmp-target <triple> --;
    • an entry in command form;
    • an -Xarch_<arch> value read as a bare option.
  • Through libclang:
    • an entry with -I -- and -- case.cpp other.cpp is parsed with the header the joined -I-- finds;
    • -Xarch_host -MD writes no .d file.
  • tests/test_builtin_headers.py: -mllvm hands its argument on, checked against the driver.
  • tests/test_preproc_includer.py: an environment at env_version 6 is read again.

Checklist

  • I wrote this change myself and have read every line of it; it was not generated automatically from an issue.
  • I ran the package's tests (uv run poe test-<package>) and they pass.
  • Documentation is updated where behaviour or options change.
  • The package's docs/changelog.rst has an entry under Unreleased.
  • uv run poe lint and uv run poe typecheck pass.

Two shared extraction cases for the libclang engine: an entry that ends its
options with `--` before its input (only the active arms' markers are
extracted), and an entry whose `--` is the value of -Xlinker (the -DFEATURE
after it is still read as an option). Both fail against the current code: the
first extracts the markers of every arm, the second loads no translation unit
and extracts nothing.
… only there

filter_args leaves an options-ending `--` out with every argument after it
(they are inputs), so the arguments the libclang engine adds are no longer
read as inputs and the parse keeps its preprocessing record.

A `--` that is an option's value ends nothing. libclang, though, takes the
first `--` of its arguments for the end of the options whatever precedes it,
so none is passed on: a pass-through option (-Xlinker, -Xclang, -mllvm,
-Xarch_<arch>, -Xopenmp-target=<triple>, ...) whose value is `--` is left out
with it, and a -I/-include/-isystem/-iquote/-idirafter/-isysroot/-x whose value
is `--` is joined to it (-I--), which the driver reads the same. Any other
pass-through value is kept verbatim with its option.

filter_args asks builtin_headers.passes_next_argument, the predicate the
-resource-dir placement uses, so there is one list of pass-through options;
-mllvm joins it. env_version 7: records extracted with such an entry change.
The driver reads the value of -Xarch_host (and of -Xarch_<arch> for the
architecture a Mach-O target compiles) as one of its own options, so
filter_args routes it through the rules a bare option goes through: a value
that would be left out (-MD, -MMD, -c, -o out.o, -MF dep.d, the input) is left
out with its -Xarch_*. Passed on, -Xarch_host -MD made the parse write a
dependency file into the working directory.

The filter_args docstring and the docs now say exactly which options' `--`
value ends nothing, and why leaving a pass-through option out with its `--`
loses nothing: a real compile rejects a `--` handed to the compiler or the
driver, and the other tools have no say in preprocessing. Two tests run the
joined -I-- and the inputs after an entry's `--` through libclang.
@github-actions github-actions Bot added the pkg: sphinx-codelinks Concerns the sphinx-codelinks package (packages/sphinx-codelinks) label Nov 12, 2026
@read-the-docs-community

Copy link
Copy Markdown

Documentation build overview

📚 sphinx-codelinks | 🛠️ Build #35853909 | 📁 Comparing fe70a10 against latest (ed810e7)

  🔍 Preview build  

4 files changed
± changelog.html
± components/analyse.html
± development/traceability.html
± src/sphinx_codelinks/analyse/preproc/compile_db.html

@codecov

codecov Bot commented Nov 12, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.11%. Comparing base (ed810e7) to head (fe70a10).

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #2510   +/-   ##
=======================================
  Coverage   95.10%   95.11%           
=======================================
  Files         161      161           
  Lines       31392    31415   +23     
=======================================
+ Hits        29856    29880   +24     
+ Misses       1536     1535    -1     
Flag Coverage Δ
ai-index 100.00% <ø> (ø)
codelinks 96.15% <100.00%> (+0.02%) ⬆️
mounts 94.40% <ø> (ø)
pytests 94.88% <ø> (ø)
reports 97.02% <ø> (ø)
ub-directives 95.79% <ø> (ø)
ub-test-reports 91.14% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@chrisjsewell
chrisjsewell merged commit ffea42a into master Nov 13, 2026
48 checks passed
@chrisjsewell
chrisjsewell deleted the claude/lucid-hawking-pa89x2-dashdash branch November 13, 2026 08:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pkg: sphinx-codelinks Concerns the sphinx-codelinks package (packages/sphinx-codelinks)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant