Skip to content

fix: replace control characters that XML 1.0 forbids when writing - #204

Open
emschwartz wants to merge 1 commit into
rust-syndication:masterfrom
emschwartz:replace-control-chars
Open

emschwartz wants to merge 1 commit into
rust-syndication:masterfrom
emschwartz:replace-control-chars

Conversation

@emschwartz

Copy link
Copy Markdown
Contributor

A control character in any text or attribute value (here U+001E in a title) is written raw, which makes the whole document invalid XML:

before: <title>Chan<0x1E>nel</title>   (xmllint: "invalid Char value 30")
after:  <title>Chan�nel</title>

XML 1.0 forbids U+0000-U+001F other than tab, LF and CR even as &#x1E;, so they can't be escaped. This wraps the output writer in Channel::write_to and pretty_write_to so each such byte is written as U+FFFD instead (a replacement rather than a deletion, so neighbouring characters can't join, e.g. ]]\x01> would otherwise close a CDATA section). It covers text, CDATA, attributes and extensions in one place. U+FFFE/U+FFFF are not handled.

Behavior change: round-tripping through rss's own reader now yields U+FFFD where a raw \x01 used to come back intact.

Tested with test_control_characters_replaced (title, CDATA content, enclosure URL attribute; parses the output back). Also ran the CI commands: cargo test --all-features, cargo clippy --all --all-features -- -D warnings, cargo fmt --all -- --check, cargo doc --features "validation serde".

Test output before the fix
thread 'test_control_characters_replaced' panicked at tests/write.rs:517:5:
assertion failed: !output.contains(|c|
            matches!(c, '\0'..='\u{8}' | '\u{B}' | '\u{C}' |
                '\u{E}'..='\u{1F}'))

Authored by Claude (claude-sonnet-5-5, claude-opus-5-5)

Characters such as U+001E or U+0008 in a title were written raw, which
conforming parsers reject for the whole document. They cannot be
represented even as character references, so write U+FFFD instead. A
replacement rather than a deletion keeps the neighbouring characters
from joining into new markup, e.g. `]]\u{1}>` ending a CDATA section.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@emschwartz
emschwartz force-pushed the replace-control-chars branch from 68ec90c to 3d2419f Compare November 11, 2026 16:27
@codecov

codecov Bot commented Nov 11, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 88.23529% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 82.67%. Comparing base (8598600) to head (3d2419f).
⚠️ Report is 14 commits behind head on master.

Files with missing lines Patch % Lines
src/toxml.rs 84.61% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #204      +/-   ##
==========================================
+ Coverage   80.98%   82.67%   +1.68%     
==========================================
  Files          23       23              
  Lines        1530     1558      +28     
==========================================
+ Hits         1239     1288      +49     
+ Misses        291      270      -21     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@DCjanus
DCjanus self-requested a review November 12, 2026 06:42
@andy128k

Copy link
Copy Markdown
Member

You could contribute this change to quick_xml crate.

@DCjanus
DCjanus removed their request for review November 16, 2026 03:01
@emschwartz

Copy link
Copy Markdown
Contributor Author

Hi @andy128k, I looked into contributing it to quick-xml instead. The maintainer previously said that "silently skip[ping] invalid characters" is an "anti-pattern" (tafia/quick-xml#496 (comment)). They've also considered emitting an error or just allowing the invalid characters.

Either way, it seems that the handling of these characters would end up being up to the user of that library, so I think it would make sense to pick a specific policy (like replacing the characters) in this crate.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants