Draft
Conversation
This PR introduces a native C parser and bytecode compiler for Liquid control flow tags, providing significant performance improvements for template parsing and rendering. ## Performance Improvements - Parsing: 2-10x faster depending on template complexity - Control flow rendering: ~2x faster for if/case statements - Variable-heavy templates: up to 10x faster parsing ## Architecture ### New Components - **Arena Allocator** (arena.c/h): Fast bump-pointer allocation for AST nodes - **AST Structures** (ast.c/h): 21 node types for all Liquid constructs - **Template Parser** (template_parser.c/h): Recursive descent parser - **Code Generator** (codegen.c/h): AST to bytecode compiler ### New VM Opcodes Control flow: - OP_JUMP, OP_JUMP_IF_FALSE, OP_JUMP_IF_TRUE (with wide variants) Comparisons: - OP_CMP_EQ, OP_CMP_NE, OP_CMP_LT, OP_CMP_GT, OP_CMP_LE, OP_CMP_GE - OP_CMP_CONTAINS Variables: - OP_ASSIGN, OP_INCREMENT, OP_DECREMENT Loop control: - OP_FOR_INIT, OP_FOR_NEXT, OP_FOR_CLEANUP - OP_BREAK, OP_CONTINUE ### What's Native vs Ruby **Native C execution:** - if/elsif/else/unless - native jump opcodes - case/when/else - native comparison + jumps - assign - native OP_ASSIGN - increment/decrement - native opcodes - All comparison operators **Still uses Ruby (by design):** - Custom tags (extensibility) - Filters (Ruby filter methods) - render/include tags - Drops and complex objects ## Bug Fixes - Fixed blank keyword comparison (x == blank) - Fixed empty keyword comparison (x == empty) - Fixed nil contains check (array contains nil) - Fixed Bus Error crash in case/when with multiple branches - Fixed break/continue in for loops ## Tests - 272 unit tests passing - 187 new tests for parser/VM functionality - 91.5% liquid-spec conformance (108/118)
str_forloop was declared but never assigned the "forloop" string, causing OP_FOR_INIT/NEXT/CLEANUP to use Qnil as a hash key instead of "forloop" when managing forloop state in context scope. This could cause undefined behavior when: - Running native for loops - Mixing C and Ruby templates with for loops - GC occurring during for loop execution
- Update Liquid gem to 5.11.0 which adds Liquid::Utils.to_s method - Update parse_expression to accept new safe: keyword argument - Fix adapter loading order to define Liquid::Environment shim early - Fix strict_errors vs strict_variables confusion in render block - Use static_environments instead of environments for global data All 710 liquid-spec tests now pass with max complexity 400/400.
…perties Changes: - Fix null/nil comparisons to return false instead of raising ArgumentError - Add to_liquid_value unwrapping for drop comparisons and truthiness checks - Add string property access for .first and .last (returns first/last char) - Add to_liquid method to Blank and Empty classes Results: 4060/4102 tests passing (98.98%) - Basics: 710/710 (100%) - Liquid Ruby: 1518/1543 (98.4%) - Shopify Production Recordings: 1832/1849 (99.1%) Remaining 42 failures are mostly Shopify-specific case statement quirks (multiple else clauses, when after else, fall-through) that would require significant parser changes to support without breaking normal behavior.
Implements Shopify's quirky case statement behavior: - When tags after else are now allowed - Multiple else clauses all execute (until a when matches) - Multiple matching when clauses all execute (fall-through) Results: 4067/4102 tests passing (99.15%) - Basics: 710/710 (100%) - Liquid Ruby: 1521/1543 (98.6%) - Shopify Production Recordings: 1836/1849 (99.3%)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR introduces a native C parser and bytecode compiler for Liquid control flow tags, providing significant performance improvements.
Performance Improvements
New Components
arena.c/h): Fast bump-pointer allocationast.c/h): 21 node types for all Liquid constructstemplate_parser.c/h): Recursive descent parsercodegen.c/h): AST → bytecode compilerNew VM Opcodes (15+)
OP_JUMP,OP_JUMP_IF_FALSE,OP_JUMP_IF_TRUEOP_CMP_EQ/NE/LT/GT/LE/GE/CONTAINSOP_ASSIGN,OP_INCREMENT,OP_DECREMENTOP_FOR_INIT,OP_FOR_NEXT,OP_BREAK,OP_CONTINUENative vs Ruby
Bug Fixes
blankkeyword comparisonemptykeyword comparisonnilcontains checkTest plan