Skip to content

Add :max_depth option to from_document/2 and from_fragment/2 - #43

Merged
jonatanklosko merged 2 commits into
dashbitco:mainfrom
ericmj:max-depth-option
Oct 18, 2026
Merged

jonatanklosko merged 2 commits into
dashbitco:mainfrom
ericmj:max-depth-option

Conversation

@ericmj

@ericmj ericmj commented Oct 16, 2026

Copy link
Copy Markdown
Contributor

lexbor walks the stack of open elements for many tokens during tree construction, for example to look for a p element in button scope on every div or ul start tag, and for most end tags. Input that nests elements thousands deep takes time quadratic in its size to parse: nested <div> takes 225 ms at 100 KB and 964 ms at 200 KB with the pinned lexbor (Apple M5 Max). lexbor/lexbor#437 fixes the p check upstream, and the end tags remain.

:max_depth raises once more elements than the limit are open, where <html> is at depth 1 for documents and the top-level nodes are at depth 1 for fragments. It wraps the tree builder's token callback, so it doesn't need a lexbor change. To install the callback, from_document and from_fragment now call lexbor's chunk functions, which is what lxb_html_document_parse() and lxb_html_document_parse_fragment() do internally.

Without the option the output doesn't change: 200,000 random tag-soup documents and 200,000 fragments produce the same HTML as main, and the same holds with max_depth: 10_000. With max_depth: 512, 100 KB of nested <div> raises after 0.1 ms, and inputs that stay under the limit, such as 500 nested <span> followed by 25,000 </x>, parse in under 10 ms. Ordinary HTML parses at the same speed with or without the option.

lexbor's tree construction walks the stack of open elements for many
tokens, so input that nests elements thousands deep takes time quadratic
in its size to parse. The option stops parsing and raises once more
elements than that are open.
@ericmj
ericmj marked this pull request as ready for review October 16, 2026 17:23
Comment thread c_src/lazy_html.cpp Resolved
Comment thread c_src/lazy_html.cpp Resolved
Comment thread c_src/lazy_html.cpp Resolved
Co-authored-by: Jonatan Kłosko <jonatanklosko@gmail.com>
@jonatanklosko
jonatanklosko merged commit 424bc06 into dashbitco:main Oct 18, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants