Fix markdown_parser: unclosed <u> markers no longer delete surrounding text - #15275
Conversation
…g text An unclosed `<u>` marker was being tokenized as a normal emphasis delimiter and matched against a later `<u>` marker by `process_emphasis` (since both share DelimiterKind::UnderlineStart and can_open_for only checked kind equality). This silently consumed both delimiters and the text nodes between them. `<u>` is only ever meant to be closed by an explicit `</u>`, which is handled separately by parse_underline (it never goes through can_open_for). Pairing two `<u>` openers via the emphasis algorithm is therefore always incorrect. Fix can_open_for to reject DelimiterKind::UnderlineStart openers, so an unclosed <u> now round-trips to literal text, consistent with how an unclosed * or ~~ is handled. Fixes warpdotdev#12863 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
Thank you for your pull request and welcome to our community. We require contributors to sign our Contributor License Agreement, and we don't seem to have the users @arpansahu on file. In order for us to review and merge your code, each contributor must visit https://cla.warp.dev to read and agree to our CLA. Once you have done so, please comment |
|
I'm starting a first review of this pull request. You can view the conversation on Warp. I reviewed this pull request and requested human review from: Comment Powered by Oz |
There was a problem hiding this comment.
Overview
This PR updates markdown_parser so bare <u> start tags are not matched as opener/closer pairs by the emphasis algorithm; underline remains closed only by explicit </u>. It also adds focused unit coverage for the reported unclosed-underline cases.
Concerns
- None found.
Verdict
Found: 0 critical, 0 important, 0 suggestions
Approve
Comment /oz-review on this pull request to retrigger a review (up to 3 times on the same pull request).
Powered by Oz
Description
parse_markdownwas silently deleting text when a<u>underline marker was left unclosed and another<u>appeared later in the same line.<u>is tokenized as anUnderlineStartdelimiter and pushed onto the shared delimiter stack.process_emphasistreats two<u>markers as a matchable opener/closer pair (sincecan_open_foronly checkedkindequality), consuming both markers and removing the text nodes between them.Underline is only ever supposed to be closed by an explicit
</u>, which is handled separately byparse_underline(it never goes throughcan_open_for). Pairing two<u>openers through the emphasis path is therefore always wrong.Fix
can_open_fornow explicitly rejectsDelimiterKind::UnderlineStartopeners, so an unclosed<u>round-trips to literal text — the same behavior as an unclosed*or~~.<u><u><u><u><u>a<u>a(both<u>deleted)<u>a<u>a <u>word<u> ba word b(both<u>deleted)a <u>word<u> bLinked Issue
Fixes #12863
ready-to-implement.crates/markdown_parser.Testing
test_parse_unclosed_underline_round_trips_to_literal_text, covering the three repro cases from the issue (<u><u>,<u>a<u>,a <u>word<u> b).cargo test -p markdown_parser: all 155 tests pass (154 pre-existing + 1 new).cargo fmt -p markdown_parser --check: clean.cargo clippy -p markdown_parser --all-targets --all-features -- -D warnings: no warnings.This is a crate-scoped, platform-independent parser fix with no UI surface, so it was verified via the automated test suite above rather than manual app testing (
./script/runbuilds the full desktop app, which isn't feasible in this environment).Screenshots / Videos
N/A — parser-only change, no UI.
Agent Mode