v0.11.0: UseAnchoredLiteral 32-133x speedup (#79) #96
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Implements dual-NFA compilation for patterns with '.': - UTF-8 NFA: handles all valid UTF-8 codepoints (~28 states per '.') - ASCII NFA: optimized for ASCII-only input (1-2 states per '.') At runtime, input is checked using SIMD (AVX2 on x86-64) to determine if all bytes are ASCII. If so, the faster ASCII NFA is used. Performance for Issue #79 pattern ^/.*[\w-]+\.php: - Short input (6B): 3.7x faster - Medium input (23B): 2.3x faster - Long input (49B): 1.5x faster SIMD isASCII throughput: 20-41 GB/s (AVX2), 10+ GB/s (SWAR fallback) NFA state reduction: 6x for single '.', 2.8x for Issue #79 pattern New files: - simd/ascii_*.go, simd/ascii_amd64.s: SIMD ASCII detection - nfa/pattern_analysis.go: ContainsDot() helper - meta/ascii_optimization_test.go: integration tests - nfa/compile_ascii_test.go: NFA compilation tests Config: EnableASCIIOptimization (default: true)
ExtractSuffixes was returning empty for patterns like '\.php$' because the last AST element is OpEndLine ($), not the literal. Now skips trailing anchors (OpEndLine, OpEndText) to find the actual suffix literal. Issue #79 improvements: - Wrong suffix: 2017ns → 15ns (91x faster than stdlib!) - Wrong prefix: already fast at 5.2ns (13x faster than stdlib) - Matching: 972ns vs stdlib 788ns (1.23x slower, was 5.3x)
epsilonClosureOnePass was not handling StateLook states, causing OnePass DFA to fail for all patterns with explicit anchors. Now treats anchor states as epsilon transitions: - Start anchors (^, \A): Always satisfied in anchored mode - End anchors ($, \z): Follow epsilon; match verified at input end Simple anchored patterns like ^abc$ now work correctly. Patterns with .* are still rejected (inherent ambiguity).
Add specialized O(1) matching for ^prefix.*suffix$ patterns: - Pattern detection via AST analysis (DetectAnchoredLiteral) - Fast matching with O(k) prefix/suffix checks (MatchAnchoredLiteral) - 256-byte lookup table for charclass bridge verification - Proper UTF-8 handling (reject non-ASCII in ASCII-only charclass) Benchmark results vs stdlib for ^/.*[\w-]+\.php$: - Short input: 24x faster (13 ns vs 322 ns) - Medium input: 28x faster (14 ns vs 399 ns) - Long input: 51x faster (10 ns vs 489 ns) - No match: 99x faster (7 ns vs 689 ns) Issue #79 resolution: from 5.3x SLOWER to 24-99x FASTER than stdlib.
- engine.go (230): Engine struct, Stats, core API - compile.go (526): Compilation, builders - find.go (749): Find methods returning *Match - find_indices.go (733): Zero-alloc FindIndices methods - ismatch.go (353): IsMatch boolean methods - findall.go (285): FindAll*, Count, FindSubmatch - meta.go (81): Package documentation only Also fixes linter issues in anchored_literal.go
- CHANGELOG.md: v0.11.0 release notes with UseAnchoredLiteral, ASCII detection - README.md: 17 strategies, added AnchoredLiteral to strategy table - ROADMAP.md: v0.11.0 current, v0.12.0 CompositeSearcher next - OPTIMIZATIONS.md: Added AnchoredLiteral as 9th key optimization
Benchmark ComparisonComparing Summary:
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Release v0.11.0 - resolves Issue #79 (UTF-8 dot performance regression).
New Features
UseAnchoredLiteral strategy - O(1) specialized matching for
^prefix.*suffix$patterns^/.*[\w-]+\.php$: 32-133x faster than stdlib (was 5.3x slower)V11-002 ASCII runtime detection - SIMD-accelerated input classification
.Bug Fixes
^,$,\A,\z)Internal
Performance (Issue #79 pattern)
Test plan
go test ./...passesgo test -race ./...passesgolangci-lint run- 0 issuesCloses #79