Skip to content

Upgrade MARISA library from 0.2.6 to 0.3.1#1078

Open
BYVoid wants to merge 10 commits intomasterfrom
claude/update-marisa-dependency-eMUHT
Open

Upgrade MARISA library from 0.2.6 to 0.3.1#1078
BYVoid wants to merge 10 commits intomasterfrom
claude/update-marisa-dependency-eMUHT

Conversation

@BYVoid
Copy link
Copy Markdown
Owner

@BYVoid BYVoid commented Mar 27, 2026

Summary

Update the bundled marisa-trie dependency from 0.2.6 to 0.3.1.

Changes

  • Replaced deps/marisa-0.2.6 with deps/marisa-0.3.1 (downloaded from https://github.com/s-yata/marisa-trie/releases/tag/v0.3.1)
  • Updated path references in CMakeLists.txt, src/CMakeLists.txt, and node/node_opencc.gypi
  • Replaced upstream CMakeLists.txt with a minimal subdirectory-compatible version (original preserved as CMakeLists.txt.upstream)
  • Fixed mingw build: defined _WIN32_WINNT=0x0602 in mapper.cc before <windows.h> for PrefetchVirtualMemory / WIN32_MEMORY_RANGE_ENTRY
  • Fixed MSVC x86 build: replaced _BitScanForward64 with two _BitScanForward calls on 32-bit targets in bit-vector.cc
  • Added TODO in MODULE.bazel to update the Bazel dependency when marisa-trie 0.3.1 becomes available on the Bazel Central Registry

Notes

  • The Bazel MODULE.bazel dependency remains at 0.2.6 because 0.3.1 is not yet published to the Bazel Central Registry.

Binary compatibility

Verified that marisa 0.3.1 can read .ocd2 dictionaries serialized by marisa 0.2.6. Built all 16 dictionaries with 0.2.6, then ran the full test suite against them using 0.3.1 — all 16 tests passed. The trie binary format is unchanged between versions.

Replace deps/marisa-0.2.6 with marisa-trie v0.3.1 downloaded from
https://github.com/s-yata/marisa-trie/releases/tag/v0.3.1 and update
all build system references (CMakeLists.txt, node_opencc.gypi).

https://claude.ai/code/session_01Lw2Sjg4RgnMG6PuDoKRay4
@BYVoid BYVoid force-pushed the claude/update-marisa-dependency-eMUHT branch from 18699d9 to 6aefa5c Compare March 27, 2026 05:17
claude added 5 commits March 27, 2026 05:18
The upstream marisa-0.3.1 CMakeLists.txt is a full standalone project
(cmake_minimum_required 3.22, project(), tools, tests, install rules)
which conflicts when used via add_subdirectory() from OpenCC (which
requires CMake 3.5). Replace it with a minimal version matching the
old 0.2.6 style that just builds a static library. The original is
preserved as CMakeLists.txt.upstream.

https://claude.ai/code/session_01Lw2Sjg4RgnMG6PuDoKRay4
marisa 0.3.1 uses PrefetchVirtualMemory and WIN32_MEMORY_RANGE_ENTRY
in mapper.cc, which require _WIN32_WINNT >= Windows 8. Add the
compile definition for MINGW builds.

https://claude.ai/code/session_01Lw2Sjg4RgnMG6PuDoKRay4
_BitScanForward64 is not available on 32-bit MSVC targets (Win32/x86).
Fall back to two _BitScanForward calls on the low and high 32-bit
halves of the uint64_t.

https://claude.ai/code/session_01Lw2Sjg4RgnMG6PuDoKRay4
The compile definition in CMakeLists.txt may not take effect early
enough. Define _WIN32_WINNT=0x0602 (Windows 8) directly in mapper.cc
before #include <windows.h> to ensure PrefetchVirtualMemory and
WIN32_MEMORY_RANGE_ENTRY are available.

https://claude.ai/code/session_01Lw2Sjg4RgnMG6PuDoKRay4
BYVoid and others added 4 commits March 27, 2026 17:30
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Revert mapper.cc to upstream 0.3.1 and conditionally compile the
PrefetchVirtualMemory call so it builds on older Windows targets.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@frankslin
Copy link
Copy Markdown
Collaborator

@codex review

Repository owner deleted a comment from chatgpt-codex-connector bot Mar 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants