Skip to content

Create Shared Test Suite#38

Open
brenank wants to merge 12 commits intoVestaboard:mainfrom
brenank:brenank/improve-tests
Open

Create Shared Test Suite#38
brenank wants to merge 12 commits intoVestaboard:mainfrom
brenank:brenank/improve-tests

Conversation

@brenank
Copy link
Copy Markdown

@brenank brenank commented Mar 19, 2026

Creates a common cross language test suite to help catch behavior drift between languages
Only affects test logic, does not change any production code.

Moves common tests to live under /test

  • /test/inputs - contains all test cases
  • /test/expected - test expected values for each test case (named the same)
  • /test/language-exceptions/<language>/ - same as expected, but overrides for specific expected values for a language (used to describe deviations/drift from common behavior)

All language tests now utilize/import these common test cases, and run them in their respective languages, rather than reimplementing the entire suite.

I know this is a large change, so feel free to close if this is not desired, or is too much to review.

@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Mar 19, 2026

Important

Review skipped

Too many files!

This PR contains 300 files, which is 150 over the limit of 150.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 589f0f3c-8cf1-4964-87a9-68f7e240cbd0

📥 Commits

Reviewing files that changed from the base of the PR and between e4f583b and 78c54c1.

📒 Files selected for processing (300)
  • .github/workflows/conformance.yml
  • package.json
  • php/tests/CalendarTest.php
  • php/tests/CharacterCodesToAsciiTest.php
  • php/tests/CharacterCodesToStringTest.php
  • php/tests/ClassicTest.php
  • php/tests/Conformance/CalendarConformanceTest.php
  • php/tests/Conformance/CharacterCodesToAsciiConformanceTest.php
  • php/tests/Conformance/CharacterCodesToStringConformanceTest.php
  • php/tests/Conformance/ClassicConformanceTest.php
  • php/tests/Conformance/ConformanceTestCase.php
  • php/tests/Conformance/HasSpecialCharactersConformanceTest.php
  • php/tests/Conformance/ParseComponentConformanceTest.php
  • php/tests/Conformance/SanitizeSpecialCharactersConformanceTest.php
  • php/tests/Conformance/VbmlConformanceTest.php
  • php/tests/HasSpecialCharactersTest.php
  • php/tests/SanitizeSpecialCharactersTest.php
  • php/tests/VbmlTest.php
  • python/tests/conformance/__init__.py
  • python/tests/conformance/support.py
  • python/tests/conformance/test_calendar.py
  • python/tests/conformance/test_character_codes_to_ascii.py
  • python/tests/conformance/test_character_codes_to_string.py
  • python/tests/conformance/test_classic.py
  • python/tests/conformance/test_has_special_characters.py
  • python/tests/conformance/test_parse_component.py
  • python/tests/conformance/test_sanitize_special_characters.py
  • python/tests/conformance/test_vbml.py
  • python/tests/test_calendar.py
  • python/tests/test_character_codes.py
  • python/tests/test_character_codes_to_ascii.py
  • python/tests/test_character_codes_to_string.py
  • python/tests/test_classic.py
  • python/tests/test_copy_character_codes.py
  • python/tests/test_create_empty_board.py
  • python/tests/test_has_special_characters.py
  • python/tests/test_horizontal_align.py
  • python/tests/test_parse_component.py
  • python/tests/test_random_colors.py
  • python/tests/test_sanitize_special_characters.py
  • python/tests/test_vbml.py
  • python/tests/test_vertical_align.py
  • scripts/validate-conformance-fixtures.mjs
  • src/__tests__/calendar.spec.ts
  • src/__tests__/characterCodesToAscii.spec.ts
  • src/__tests__/characterCodesToString.spec.ts
  • src/__tests__/classic.spec.ts
  • src/__tests__/conformance/calendar.spec.ts
  • src/__tests__/conformance/characterCodesToAscii.spec.ts
  • src/__tests__/conformance/characterCodesToString.spec.ts
  • src/__tests__/conformance/classic.spec.ts
  • src/__tests__/conformance/hasSpecialCharacters.spec.ts
  • src/__tests__/conformance/parseComponent.spec.ts
  • src/__tests__/conformance/sanitizeSpecialCharacters.spec.ts
  • src/__tests__/conformance/support.ts
  • src/__tests__/conformance/vbml.spec.ts
  • src/__tests__/hasSpecialCharacters.spec.ts
  • src/__tests__/parseComponent.spec.ts
  • src/__tests__/randomColors.spec.ts
  • src/__tests__/sanitizeSpecialCharacters.spec.ts
  • src/__tests__/vbml.spec.ts
  • test/README.md
  • test/expected/calendar/applies_default_day_color_to_visible_days.json
  • test/expected/calendar/creates_calendar_with_four_weeks.json
  • test/expected/calendar/does_not_highlight_days_outside_of_month.json
  • test/expected/calendar/hides_dates.json
  • test/expected/calendar/hides_month_year.json
  • test/expected/calendar/hides_weekday_header.json
  • test/expected/calendar/renders_single_date_header_when_last_week_has_one_day.json
  • test/expected/calendar/renders_single_date_header_when_month_starts_on_saturday.json
  • test/expected/characterCodesToAscii/converts_color_codes.json
  • test/expected/characterCodesToAscii/handles_multiple_rows.json
  • test/expected/characterCodesToAscii/renders_filled_cells_on_white_board.json
  • test/expected/characterCodesToAscii/renders_filled_cells_with_default_board_color.json
  • test/expected/characterCodesToAscii/spaces_out_letters.json
  • test/expected/characterCodesToString/converts_two_line_sentence.json
  • test/expected/characterCodesToString/converts_word_to_string.json
  • test/expected/characterCodesToString/does_not_insert_line_break_when_first_word_fits.json
  • test/expected/characterCodesToString/handles_breaks.json
  • test/expected/characterCodesToString/handles_line_breaks.json
  • test/expected/characterCodesToString/ignores_color_codes.json
  • test/expected/characterCodesToString/returns_empty_string_for_breakable_rows_only.json
  • test/expected/characterCodesToString/treats_filled_cells_as_spaces.json
  • test/expected/classic/converts_ae_umlaut.json
  • test/expected/classic/converts_double_newline.json
  • test/expected/classic/converts_embedded_char_code_string.json
  • test/expected/classic/converts_emoji_colors.json
  • test/expected/classic/converts_hyphen_string.json
  • test/expected/classic/converts_long_string_with_digits.json
  • test/expected/classic/converts_longer_string.json
  • test/expected/classic/converts_single_char_code.json
  • test/expected/classic/converts_single_newline.json
  • test/expected/classic/converts_special_characters.json
  • test/expected/classic/converts_string_to_classic_board.json
  • test/expected/classic/preserves_triple_spaces.json
  • test/expected/classic/respects_double_spaces.json
  • test/expected/classic/returns_empty_board_for_empty_string.json
  • test/expected/hasSpecialCharacters/excludes_black_color_swatch.json
  • test/expected/hasSpecialCharacters/excludes_ios_double_quote.json
  • test/expected/hasSpecialCharacters/excludes_ios_single_quote.json
  • test/expected/hasSpecialCharacters/excludes_newlines.json
  • test/expected/hasSpecialCharacters/excludes_orange_color_swatch.json
  • test/expected/hasSpecialCharacters/excludes_white_color_swatch.json
  • test/expected/hasSpecialCharacters/includes_fractions.json
  • test/expected/hasSpecialCharacters/returns_false_for_empty_string.json
  • test/expected/hasSpecialCharacters/returns_false_for_lowercase_alphabet.json
  • test/expected/hasSpecialCharacters/returns_false_for_numbers.json
  • test/expected/hasSpecialCharacters/returns_false_for_supported_symbols.json
  • test/expected/hasSpecialCharacters/returns_false_for_uppercase_alphabet.json
  • test/expected/hasSpecialCharacters/returns_true_for_mixed_special_and_standard_characters.json
  • test/expected/hasSpecialCharacters/returns_true_for_special_character.json
  • test/expected/parseComponent/adds_extra_spaces.json
  • test/expected/parseComponent/adds_template_props.json
  • test/expected/parseComponent/allows_array_iteration.json
  • test/expected/parseComponent/allows_conditions_when_false.json
  • test/expected/parseComponent/allows_conditions_when_true.json
  • test/expected/parseComponent/allows_newlines.json
  • test/expected/parseComponent/allows_newlines_after_spaces.json
  • test/expected/parseComponent/allows_newlines_before_spaces.json
  • test/expected/parseComponent/automatically_breaks_line.json
  • test/expected/parseComponent/breaks_on_lines_with_character_codes.json
  • test/expected/parseComponent/converts_emoji_characters_to_character_codes.json
  • test/expected/parseComponent/does_not_break_when_unnecessary.json
  • test/expected/parseComponent/formats_longer_message_centered.json
  • test/expected/parseComponent/formats_longer_plain_text.json
  • test/expected/parseComponent/formats_plain_text.json
  • test/expected/parseComponent/horizontally_aligns_center.json
  • test/expected/parseComponent/horizontally_aligns_justified.json
  • test/expected/parseComponent/horizontally_aligns_right.json
  • test/expected/parseComponent/horizontally_and_vertically_aligned_center.json
  • test/expected/parseComponent/justified_long_complex_message.json
  • test/expected/parseComponent/justified_when_flowing_to_next_line.json
  • test/expected/parseComponent/justified_when_full_line_covered.json
  • test/expected/parseComponent/parses_absolute_component.json
  • test/expected/parseComponent/parses_character_codes.json
  • test/expected/parseComponent/parses_raw_component.json
  • test/expected/parseComponent/parses_two_digit_character_codes.json
  • test/expected/parseComponent/splits_long_words.json
  • test/expected/parseComponent/throws_for_invalid_character_codes.json
  • test/expected/parseComponent/vertically_aligns_bottom.json
  • test/expected/parseComponent/vertically_aligns_center.json
  • test/expected/parseComponent/vertically_aligns_center_multiple_rows.json
  • test/expected/parseComponent/vertically_aligns_center_sticks_to_top_when_no_even_padding.json
  • test/expected/sanitizeSpecialCharacters/accepts_whitespace_after_heart.json
  • test/expected/sanitizeSpecialCharacters/converts_cedilla.json
  • test/expected/sanitizeSpecialCharacters/converts_ellipsis.json
  • test/expected/sanitizeSpecialCharacters/converts_en_dash.json
  • test/expected/sanitizeSpecialCharacters/converts_f_hook.json
  • test/expected/sanitizeSpecialCharacters/converts_fraction_slash.json
  • test/expected/sanitizeSpecialCharacters/converts_german_lowercase_ae.json
  • test/expected/sanitizeSpecialCharacters/converts_german_lowercase_oe.json
  • test/expected/sanitizeSpecialCharacters/converts_german_lowercase_ue.json
  • test/expected/sanitizeSpecialCharacters/converts_german_uppercase_ae.json
  • test/expected/sanitizeSpecialCharacters/converts_german_uppercase_oe.json
  • test/expected/sanitizeSpecialCharacters/converts_german_uppercase_ue.json
  • test/expected/sanitizeSpecialCharacters/converts_ligature_ae.json
  • test/expected/sanitizeSpecialCharacters/converts_ligature_oe.json
  • test/expected/sanitizeSpecialCharacters/converts_micro_sign_to_space.json
  • test/expected/sanitizeSpecialCharacters/converts_scandinavian_a_ring.json
  • test/expected/sanitizeSpecialCharacters/converts_scandinavian_o_stroke.json
  • test/expected/sanitizeSpecialCharacters/converts_sharp_s.json
  • test/expected/sanitizeSpecialCharacters/converts_sharp_s_in_fussball.json
  • test/expected/sanitizeSpecialCharacters/converts_sharp_s_in_gross.json
  • test/expected/sanitizeSpecialCharacters/converts_sharp_s_in_weiss.json
  • test/expected/sanitizeSpecialCharacters/converts_unsupported_sequenced_emojis_to_whitespace.json
  • test/expected/sanitizeSpecialCharacters/does_not_modify_plain_text.json
  • test/expected/sanitizeSpecialCharacters/does_not_replace_vestaboard_heart.json
  • test/expected/sanitizeSpecialCharacters/handles_german_sharp_s_in_context.json
  • test/expected/sanitizeSpecialCharacters/handles_german_text_with_umlauts_in_context.json
  • test/expected/sanitizeSpecialCharacters/handles_heart_emoji_and_unsupported_emojis.json
  • test/expected/sanitizeSpecialCharacters/handles_mixed_special_characters.json
  • test/expected/sanitizeSpecialCharacters/handles_multiple_special_characters_together.json
  • test/expected/sanitizeSpecialCharacters/handles_sentence.json
  • test/expected/sanitizeSpecialCharacters/preserves_space_between_heart_and_emoji.json
  • test/expected/sanitizeSpecialCharacters/preserves_space_between_heart_and_latin_glyph.json
  • test/expected/sanitizeSpecialCharacters/preserves_spaces_between_hearts.json
  • test/expected/sanitizeSpecialCharacters/replaces_accented_character.json
  • test/expected/sanitizeSpecialCharacters/replaces_fractions_with_multiple_characters.json
  • test/expected/sanitizeSpecialCharacters/sanitizes_variation_selector_from_heart_emoji.json
  • test/expected/sanitizeSpecialCharacters/sanitizes_variation_selector_from_unicode_literal.json
  • test/expected/vbml/defaults_calendar_position_to_origin.json
  • test/expected/vbml/defaults_missing_absolute_x_to_origin.json
  • test/expected/vbml/defaults_missing_absolute_y_to_origin.json
  • test/expected/vbml/flows_third_component_to_next_line.json
  • test/expected/vbml/formats_ae_umlaut.json
  • test/expected/vbml/justifies_content_vertically.json
  • test/expected/vbml/justifies_content_vertically_three_chars.json
  • test/expected/vbml/layouts_absolute_components_by_relative_components.json
  • test/expected/vbml/layouts_absolute_components_with_raw_components_for_mountain_clock.json
  • test/expected/vbml/layouts_absolute_over_relative_components.json
  • test/expected/vbml/layouts_absolute_over_relative_components_standard_size.json
  • test/expected/vbml/layouts_calendar_component_for_christmas.json
  • test/expected/vbml/layouts_calendar_component_on_the_right.json
  • test/expected/vbml/layouts_calendar_component_with_other_components.json
  • test/expected/vbml/layouts_components_side_by_side.json
  • test/expected/vbml/layouts_components_vertically.json
  • test/expected/vbml/layouts_minimalist_calendar_component.json
  • test/expected/vbml/layouts_raw_components.json
  • test/expected/vbml/parses_single_component_on_a_board.json
  • test/expected/vbml/respects_double_returns.json
  • test/expected/vbml/respects_triple_returns.json
  • test/expected/vbml/returns_empty_flagship_board_for_empty_components.json
  • test/expected/vbml/truncates_absolute_component_below_board.json
  • test/expected/vbml/truncates_calendar_component_below_board.json
  • test/expected/vbml/uses_default_flagship_size_without_style.json
  • test/input/calendar/applies_default_day_color_to_visible_days.json
  • test/input/calendar/creates_calendar_with_four_weeks.json
  • test/input/calendar/does_not_highlight_days_outside_of_month.json
  • test/input/calendar/hides_dates.json
  • test/input/calendar/hides_month_year.json
  • test/input/calendar/hides_weekday_header.json
  • test/input/calendar/renders_single_date_header_when_last_week_has_one_day.json
  • test/input/calendar/renders_single_date_header_when_month_starts_on_saturday.json
  • test/input/characterCodesToAscii/converts_color_codes.json
  • test/input/characterCodesToAscii/handles_multiple_rows.json
  • test/input/characterCodesToAscii/renders_filled_cells_on_white_board.json
  • test/input/characterCodesToAscii/renders_filled_cells_with_default_board_color.json
  • test/input/characterCodesToAscii/spaces_out_letters.json
  • test/input/characterCodesToString/converts_two_line_sentence.json
  • test/input/characterCodesToString/converts_word_to_string.json
  • test/input/characterCodesToString/does_not_insert_line_break_when_first_word_fits.json
  • test/input/characterCodesToString/handles_breaks.json
  • test/input/characterCodesToString/handles_line_breaks.json
  • test/input/characterCodesToString/ignores_color_codes.json
  • test/input/characterCodesToString/returns_empty_string_for_breakable_rows_only.json
  • test/input/characterCodesToString/treats_filled_cells_as_spaces.json
  • test/input/classic/converts_ae_umlaut.json
  • test/input/classic/converts_double_newline.json
  • test/input/classic/converts_embedded_char_code_string.json
  • test/input/classic/converts_emoji_colors.json
  • test/input/classic/converts_hyphen_string.json
  • test/input/classic/converts_long_string_with_digits.json
  • test/input/classic/converts_longer_string.json
  • test/input/classic/converts_single_char_code.json
  • test/input/classic/converts_single_newline.json
  • test/input/classic/converts_special_characters.json
  • test/input/classic/converts_string_to_classic_board.json
  • test/input/classic/preserves_triple_spaces.json
  • test/input/classic/respects_double_spaces.json
  • test/input/classic/returns_empty_board_for_empty_string.json
  • test/input/hasSpecialCharacters/excludes_black_color_swatch.json
  • test/input/hasSpecialCharacters/excludes_ios_double_quote.json
  • test/input/hasSpecialCharacters/excludes_ios_single_quote.json
  • test/input/hasSpecialCharacters/excludes_newlines.json
  • test/input/hasSpecialCharacters/excludes_orange_color_swatch.json
  • test/input/hasSpecialCharacters/excludes_white_color_swatch.json
  • test/input/hasSpecialCharacters/includes_fractions.json
  • test/input/hasSpecialCharacters/returns_false_for_empty_string.json
  • test/input/hasSpecialCharacters/returns_false_for_lowercase_alphabet.json
  • test/input/hasSpecialCharacters/returns_false_for_numbers.json
  • test/input/hasSpecialCharacters/returns_false_for_supported_symbols.json
  • test/input/hasSpecialCharacters/returns_false_for_uppercase_alphabet.json
  • test/input/hasSpecialCharacters/returns_true_for_mixed_special_and_standard_characters.json
  • test/input/hasSpecialCharacters/returns_true_for_special_character.json
  • test/input/parseComponent/adds_extra_spaces.json
  • test/input/parseComponent/adds_template_props.json
  • test/input/parseComponent/allows_array_iteration.json
  • test/input/parseComponent/allows_conditions_when_false.json
  • test/input/parseComponent/allows_conditions_when_true.json
  • test/input/parseComponent/allows_newlines.json
  • test/input/parseComponent/allows_newlines_after_spaces.json
  • test/input/parseComponent/allows_newlines_before_spaces.json
  • test/input/parseComponent/automatically_breaks_line.json
  • test/input/parseComponent/breaks_on_lines_with_character_codes.json
  • test/input/parseComponent/converts_emoji_characters_to_character_codes.json
  • test/input/parseComponent/does_not_break_when_unnecessary.json
  • test/input/parseComponent/formats_longer_message_centered.json
  • test/input/parseComponent/formats_longer_plain_text.json
  • test/input/parseComponent/formats_plain_text.json
  • test/input/parseComponent/horizontally_aligns_center.json
  • test/input/parseComponent/horizontally_aligns_justified.json
  • test/input/parseComponent/horizontally_aligns_right.json
  • test/input/parseComponent/horizontally_and_vertically_aligned_center.json
  • test/input/parseComponent/justified_long_complex_message.json
  • test/input/parseComponent/justified_when_flowing_to_next_line.json
  • test/input/parseComponent/justified_when_full_line_covered.json
  • test/input/parseComponent/parses_absolute_component.json
  • test/input/parseComponent/parses_character_codes.json
  • test/input/parseComponent/parses_raw_component.json
  • test/input/parseComponent/parses_two_digit_character_codes.json
  • test/input/parseComponent/splits_long_words.json
  • test/input/parseComponent/throws_for_invalid_character_codes.json
  • test/input/parseComponent/vertically_aligns_bottom.json
  • test/input/parseComponent/vertically_aligns_center.json
  • test/input/parseComponent/vertically_aligns_center_multiple_rows.json
  • test/input/parseComponent/vertically_aligns_center_sticks_to_top_when_no_even_padding.json
  • test/input/sanitizeSpecialCharacters/accepts_whitespace_after_heart.json
  • test/input/sanitizeSpecialCharacters/converts_cedilla.json
  • test/input/sanitizeSpecialCharacters/converts_ellipsis.json
  • test/input/sanitizeSpecialCharacters/converts_en_dash.json
  • test/input/sanitizeSpecialCharacters/converts_f_hook.json
  • test/input/sanitizeSpecialCharacters/converts_fraction_slash.json
  • test/input/sanitizeSpecialCharacters/converts_german_lowercase_ae.json
  • test/input/sanitizeSpecialCharacters/converts_german_lowercase_oe.json
  • test/input/sanitizeSpecialCharacters/converts_german_lowercase_ue.json
  • test/input/sanitizeSpecialCharacters/converts_german_uppercase_ae.json
  • test/input/sanitizeSpecialCharacters/converts_german_uppercase_oe.json
  • test/input/sanitizeSpecialCharacters/converts_german_uppercase_ue.json
  • test/input/sanitizeSpecialCharacters/converts_ligature_ae.json
  • test/input/sanitizeSpecialCharacters/converts_ligature_oe.json

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Tip

CodeRabbit can enforce grammar and style rules using `languagetool`.

Configure the reviews.tools.languagetool setting to enable/disable rules and categories. Refer to the LanguageTool Community to learn more.

@brenank brenank marked this pull request as ready for review March 19, 2026 08:15
@brenank
Copy link
Copy Markdown
Author

brenank commented Mar 20, 2026

I was looking to port to golang as well, and this was a simple way to do the validation of the port.

@brenank brenank force-pushed the brenank/improve-tests branch from 35c130e to 0da5e8c Compare March 20, 2026 21:05
@brenank brenank force-pushed the brenank/improve-tests branch from 0da5e8c to 78c54c1 Compare March 20, 2026 21:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant