Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,10 @@ A lightweight HTML parser written in Rust. It borrows input bytes and offers
allocation-free bounded parsing by default, or heap-backed DOMs and CSS queries
with `std`. The Rust crate is imported as **`tl`**.

This README describes the current Git source. The source contains breaking changes since the published `0.2.0` API;
the first release containing them must be `0.3.x` or later.
See `Cargo.toml` for the version selected for the next release.
The published [0.2.0 API reference](https://docs.rs/rustedbytes-tl/0.2.0/tl/)
does not describe all APIs below. Generate current API docs with
This README describes the Git source with package version `0.3.0`. Compared
with the `0.2.0` API, it changes bounded parsing signatures and adds selector
variants; see [Updating consumers](https://github.com/RustedBytes/rustedbytes-tl/blob/master/docs/usage.md#updating-consumers).
Generate API documentation for your checkout with
`cargo doc --features entities --open`.

## Getting started
Expand Down Expand Up @@ -64,6 +63,7 @@ not implement browser layout or visibility rules. HTML parsing and text
extraction do not sanitize untrusted content.

- [Usage guide](https://github.com/RustedBytes/rustedbytes-tl/blob/master/docs/usage.md): executable examples and API behavior.
- [Compatibility reference](https://github.com/RustedBytes/rustedbytes-tl/blob/master/docs/compatibility.md): selector support, capacity errors and parser limits.
- [Development guide](https://github.com/RustedBytes/rustedbytes-tl/blob/master/docs/development.md): feature checks, docs, benchmarks and troubleshooting.

## Provenance and license
Expand Down
83 changes: 83 additions & 0 deletions docs/compatibility.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
# Compatibility and limits

This reference describes the current source. For complete method signatures,
run `cargo doc --features entities --open`; for examples, see the
[usage guide](usage.md). The crate parses HTML already supplied by the caller;
it does not fetch URLs, execute JavaScript or implement XPath.

## Choosing a build

| Contract | Default / `no_std` | `std` (also enabled by `entities`) |
| --- | --- | --- |
| HTML storage | Inline, fixed capacities | Heap-backed storage |
| `parse` | Six const capacity arguments | No const arguments required |
| DOM ownership | Borrows `&str` | Borrows `&str`, or owns a `String` through `VDomGuard` |
| DOM `query_selector` failure | `Err(ParseError::SelectorCapacityExceeded)` | `None` |
| Text extraction | Inspect raw nodes or write markup | `inner_text`; `decoded_inner_text` with `entities` |
| Structural CSS matching | Unavailable | Supported subset below |

Borrowing input avoids copying the entire HTML string. It does not mean that
`std` parsing, query construction or text extraction avoids allocations.
Neither build is a full browser DOM or an HTML sanitizer.

## CSS subset with std

| Selector | Example | Meaning |
| --- | --- | --- |
| Type, universal, ID, class | `a`, `*`, `#main`, `.item` | Match an element |
| Attribute presence | `[href]` | Attribute exists, including a valueless attribute |
| Attribute values | `[href='/']`, `[class~='item']`, `[href^='/']`, `[href$='.pdf']`, `[href*='docs']` | Equality, whitespace token, prefix, suffix, substring |
| Compound and list | `a.item[href]`, `p, a` | Intersection and union |
| Descendant and child | `main a`, `main > a` | Ancestor or direct parent |
| Adjacent and general sibling | `h2 + p`, `h2 ~ p` | Preceding element sibling |
| Position | `:first-child`, `:nth-child(2n+1)` | One-based element position; ignores text/comments |
| Negation | `a:not(.hidden)` | Exclude a matching selector |
| Descendant presence | `article:has(a)` | Require a matching descendant |

Type matching is ASCII-insensitive. CSS escapes, namespaces, attribute case
flags and other pseudo-classes are unsupported. Relative arguments such as
`:has(> a)` are rejected. Invalid queries return `None`; a valid query without
matches returns an empty iterator.

The parser rejects recursive selector calls at depth 64 and compounds with
64 or more simple components. These are implementation bounds, not a promise
that every smaller expression is valid. Structural queries construct indexes
proportional to DOM size; `:has` can scan many candidates.

For bounded builds, prefer individual tag, ID, class or universal selectors.
Compounds such as `a.item`, lists and combinators are rejected. A selector error
can mean unsupported syntax, so increasing capacity alone may not fix it.

## Bounded parse errors

The six `parse` capacities appear in the order shown below. Budgets include
text and comment nodes, not only HTML elements.

| Error | Budget or cause | Caller action |
| --- | --- | --- |
| `NodeCapacityExceeded` | First: all parsed nodes | Increase node capacity |
| `StackCapacityExceeded` | Second: parser stack entries | Increase nesting capacity |
| `RootCapacityExceeded` | Third: root nodes | Increase root capacity |
| `IdCapacityExceeded` | Fourth: tracked ID entries | Increase ID capacity or disable tracking |
| `ClassCapacityExceeded` | Fifth: tracked class storage | Increase class/node budgets or disable tracking |
| `SelectorCapacityExceeded` | Sixth: selector storage, or rejected selector syntax | Review syntax and selector budget |
| `AttributeCapacityExceeded` | 8 ordinary attributes per tag | Reduce attributes or use `std` |
| `ChildCapacityExceeded` | 256 direct children per tag | Reduce children or use `std` |
| `InvalidLength` | Input length exceeds `u32::MAX` bytes | Reject or split the input before parsing |

ID and class attributes have separate storage from the 8 ordinary attributes.
The six const arguments do not change the per-tag attribute/child bounds.
Large inline budgets also increase object and stack size. Capacity errors do
not establish that the HTML is malformed, and successful parsing does not
establish HTML5 conformance.

## HTML and output behavior

The parser does not implement the complete HTML5 tree-building algorithm.
It does not insert omitted `tbody` elements or reproduce all browser recovery
rules for misnested tags. Test selectors against your actual source HTML.

Raw text extraction concatenates descendant text without layout separators,
visibility filtering or entity decoding. Entity decoding is opt-in and does
not rewrite attributes. Serialized markup can differ from the original bytes;
retain the input when exact source formatting is needed.
12 changes: 6 additions & 6 deletions docs/development.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,9 +66,9 @@ cargo run --release --features entities --example audit_allocations

`--test` is a smoke check, not a timing measurement. `tl` benchmarks parsing;
`audit` benchmarks queries and text on a prebuilt DOM. The allocation example
measures requested allocation activity separately. See the
[audit report](rust-audit.md) for fixtures, baselines, confidence intervals and
limits. Reproduce baseline and candidate with identical toolchains, dependency
measures requested allocation activity separately. Inspect `benches/tl.rs`, `benches/audit.rs` and
`examples/audit_allocations.rs` for the actual fixtures and counters. No baseline
report is included in this checkout. Reproduce baseline and candidate with identical toolchains, dependency
resolution, features and flags; avoid concurrent builds during timing runs.

## Troubleshooting
Expand All @@ -79,7 +79,7 @@ resolution, features and flags; avoid concurrent builds during timing runs.
| `parse` requires const arguments | Enable `std`, or provide all six bounded capacities |
| `decoded_inner_text` is missing | Enable `entities`, which also enables `std` |
| Query is rejected | Check the supported CSS subset; rejection differs from an empty iterator |
| Tree differs from a browser | Review HTML5 recovery limits in the compatibility notes |
| Tree differs from a browser | Review HTML5 recovery limits in the [compatibility reference](compatibility.md) |
| Capacity error | Increase the relevant bounded budget or use heap-backed `std` parsing |

Docs-only paths trigger CI too. `ci.yml` checks code and docs. Version changes and publishing use the separate
Expand All @@ -91,8 +91,8 @@ workflows described below.
1. Run **Update version** (`update-version.yml`) from `master`. Choose `patch`,
`minor` or `major`, or enter an explicit `MAJOR.MINOR.PATCH`. The workflow
rejects malformed, unchanged or decreased versions and existing version
branches/tags. For the breaking changes since 0.2.0, choose `minor` (0.3.0)
or a later compatible release version, rather than a 0.2.x patch.
branches/tags. Choose the increment from the changes since the manifest version;
the current manifest is already `0.3.0`.
2. Review the generated `chore/version-*` PR and its explicitly dispatched
`ci.yml` run, then merge it. Only `Cargo.toml` is committed: this library
does not track `Cargo.lock`. Enable **Allow GitHub Actions to create and
Expand Down
30 changes: 28 additions & 2 deletions docs/usage.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,13 +65,38 @@ descendant, child and sibling combinators, `:first-child`, `:nth-child(an+b)`,
text and comments. Type names match ASCII-insensitively. CSS escapes,
namespaces, attribute case flags, other pseudo-classes and `:has(> a)` are
unsupported. Recursive nesting and compound size are bounded; see the
[compatibility notes](https://github.com/RustedBytes/rustedbytes-tl/blob/master/docs/silkworm-compatibility.md).
[compatibility notes](https://github.com/RustedBytes/rustedbytes-tl/blob/master/docs/compatibility.md).

Structural queries build indexes proportional to DOM node count. `:has` can
scan many candidates; it is not guaranteed to take linear time. Calling
`Selector::matches` directly lacks the structural context supplied by the DOM
query iterator.

## Walking the tree and writing markup

`dom.nodes()` includes every parsed node in source order, including text and
comments. `dom.children()` returns root handles; `tag.children().top()` returns
only the tag's direct child handles. Resolve each handle with `dom.parser()`.

```rust
# #[cfg(feature = "std")] {
let dom = tl::parse("<section><p>Hello</p><!--note--></section>", Default::default()).unwrap();
let root = dom.children()[0].get(dom.parser()).unwrap().as_tag().unwrap();
assert_eq!(root.children().top().len(), 2);
assert_eq!(dom.nodes().len(), 4);
let mut markup = String::new();
dom.write_outer_html(&mut markup).unwrap();
assert_eq!(markup, "<section><p>Hello</p><!--note--></section>");
# }
```

`write_outer_html` accepts a `core::fmt::Write` sink in either build. A `String`
sink allocates; use a bounded or streaming sink when that matters. Sink failures
return `core::fmt::Error` and can leave partial output. Serialization reconstructs
markup, so attribute quotes, ordering and other formatting can differ from the
input. It does not escape replacement attribute values or sanitize markup;
keep the original input if exact source preservation is required.

## Raw and decoded text

`inner_text` concatenates descendant text, ignores comments and leaves entities
Expand Down Expand Up @@ -132,4 +157,5 @@ APIs and makes `InlineVec::inline_parts_mut` and
raw mutable storage need review. Prefer safe collection methods such as
`get_mut`, `as_mut_slice` (vector) and `insert` (map). Do not add an unsafe block
without meeting the documented initialized-prefix and ownership invariants.
The crate version has not yet been bumped; review the Git revision you consume.
The manifest now declares `0.3.0`. Review the revision and features you consume;
`std` and bounded builds expose different parsing and query signatures.
Loading