Skip to content

Add substitution, insertion, and deletion cost functions - #81

Open
belambert wants to merge 1 commit into
mainfrom
cost-functions
Open

Add substitution, insertion, and deletion cost functions#81
belambert wants to merge 1 commit into
mainfrom
cost-functions

Conversation

@belambert

Copy link
Copy Markdown
Owner

Adds substitution_cost, insertion_cost, and deletion_cost to edit_distance(), edit_distance_backpointer(), and SequenceMatcher. Each defaults to the previous behavior, so the default path is unchanged.

test and cost stay independent: test decides equality, which drives the opcode label and the match count, while the cost functions only decide the price. substitution_cost is consulted for every aligned pair, matching ones included — that's what lets a near-match count as a match while still costing something, and it means a custom function has to handle both branches.

The boundary rows are now cumulative sums of the gap costs rather than i and j, which is what the old m == 0 / n == 0 special cases were papering over. The seq1 == seq2 shortcut is guarded on there being no custom substitution cost, since such a cost may charge for aligning an element with itself.

Everything after the two sequences is keyword-only. That matters here: the third positional slot used to be action_function, so without this a leftover edit_distance(a, b, some_action_fn) would silently be read as test instead of failing.

On highest_match_action

I need to correct something I claimed when removing it. Setting the mismatch penalty to ins + del recovers the same opcodes and the same match count — but the distance differs (6 vs 4) on the old test's input. highest_match_action accumulated unit costs while choosing by match count; a cost function uses the same numbers for both, so the distance is necessarily in the new units. test_substitution_cost_reproduces_highest_match_alignment asserts the verified values and explains why. My earlier suggestion of a penalty of 3 was also wrong for opcode-identity — above ins + del you get a different, LCS-style alignment, which is covered by its own test.

Testing

13 new unit tests and 3 new property tests; 35 total, up from 22.

  • Every expected value was computed by running the code and inspecting output, not predicted.
  • Property tests use integer costs so the assertions stay exact, and price insertions and deletions differently so each boundary is exercised on its own. They check agreement with an independently written weighted Wagner-Fischer reference (including its boundary rows), that the distance equals the summed cost of the returned alignment, and that opcodes still tile correctly under custom costs.
  • Unit tests cover both boundary sums, the substitution cap, a match that costs something, a free substitution that isn't a match, per-element gap costs, keyword-only enforcement, and SequenceMatcher reaching the cache through all three of distance(), get_opcodes(), and matches().

make ci passes, and all 35 tests pass on 3.10, 3.11, 3.12, 3.13, and 3.14 individually. sequence_matcher.py is at 100% coverage; the only uncovered lines left in edit_distance.py are pre-existing (main(), the __main__ guard, and one defensive raise).

To confirm the default path really is untouched after rewriting both DP loops, I diffed against main over 4,961 cases — exhaustive to length 4 on a 2-letter alphabet, 3,000 random integer pairs, 500 under a custom test, and 500 through the full SequenceMatcher surface including ratio() and get_matching_blocks(). Zero differences.

Three optional cost functions replace the hardcoded unit costs. Each
defaults to the previous behavior, so the default path is unchanged --
verified against main over all sequence pairs up to length four on a
two-letter alphabet plus randomized longer pairs, comparing distance,
matches, opcodes, and the whole SequenceMatcher surface.

test and cost stay independent. test decides equality, which drives the
opcode label and the match count; the cost functions only decide the
price. substitution_cost is consulted for every aligned pair, matching
ones included, which is what lets a near-match count as a match while
still costing something.

The boundary rows become cumulative sums of the gap costs rather than
i and j, which is the part the old n and m special cases were hiding.
The seq1 == seq2 shortcut is now guarded on there being no custom
substitution cost, since such a cost may charge for aligning an element
with itself.

Everything after the two sequences is keyword-only, so a stray third
positional argument raises TypeError rather than being silently read as
test -- it used to be the action function.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant