libmdbx v0.14.2 "Буревестник" (Stormy Petrel)

The frontward release with new major features and internal refactoring.

609 commits, 209 files changed, 29521 insertions(+), 15701 deletions(-)

Important:

• Due to numerous user requests, this ChangeLog will be kept in English.
  However, it should be noted that it was originally provided in Russian and then translated by AI during the formation of the release, because of this, there may be flaws in the text.

• Since 2026 libmdbx project has changed its code development and distribution model.
  To get acquainted with important changes and plans, we recommend reading the compact presentation "libmdbx: successes, obstacles, goals and roadmap", which contains important explanations in the form of embedded comments.

• The upstream of libmdbx project has been relocated to the jurisdiction of the Russian Federation.
  We are confident that this will protect the project from any sanctions and ensure its accessibility to all users around the world.
  Please use https://libmdbx.dqdkfa.ru for documentation and https://sourcecraft.dev/dqdkfa/libmdbx for the source code.
  Nonetheless libmdbx is still open source and provided with first-class free support.

Appreciations:

• Erigon for sponsorship.
• The "AntiPublic" project for sponsorship.
• Artyom Vorotnikov for Rust bindings, reporting bugs and testing.
• Stefan de Konink for Python bindings and documentation improvement.
• Cosmin Apreutesei for bug reporting and testing.
• Chloe Cano for fuzzing, bug reporing and fixes.
• Weixie Cui for bug fixing through many pull-requests.
• Alexander Kelchin (the "Serebrium" Company) for bug reporting and prototypes of exploits.
• Anton Maisak for new .NET bindings.

Backward compatibility breaks:

• The typedefs of the various callbacks are now unified and includes an asterisk of indirection in C syntax notation.
  Perhaps this is the most annoying change that breaks the builds and requires changing your code. However, it is necessary to restore order.
  In most cases, the required changes are limited to removing the * chars after a callback type(s).

• The size and composition of the MDBX_envinfo structure has been changed, and the mdbx_env_info_ex() function no longer supports the old versions. This breaks the compatibility of the ABI with older versions of the library, but preserves API compatibility at the source code level.

• The template mdbx::buffer<ALLOCATOR, POLICY> is now inherited from mdbx::slice and mdbx::buffer_tag, which simplified the C++ API and the use of the meta-programming approach.

• When building using GNU Make and CMake now, instead of a single config.h, different config-gnumake.h and config-cmake.h files are generated.

• It is forbidden to open existing tables with different flags, unless the MDBX_DB_ACCEDE option is explicitly set.

• The dumps generated by the mdbx_dump utility no longer output the current size of the database and maxreaders, so that the contents of the dump depend only on the contents of the database.

• The build option MDBX_FORCE_ASSERTIONS has been deprecated, and MDBX_CHECKING (within range of -1..3) should be used instead.

• The NDEBUG macro, which is generally accepted in C, no longer affects assert checks inside the library, but retains its traditional influence on assert checks related to argument control in the inline methods of the C++ API.

New features:

• Implemented "Early GC Cleanup".

  Now the recycled GC records are deleted not when the writing transaction is committed, but as soon as possible. This opens the way to the implementation of explicit defragmentation (without copying the database) and further to non-sequential GC processing (which will eliminate the problem of DB swelling/overflow due to GC processing stopping during long-term reading transactions).

  The amount of overhead is now proportional to the volume of operations performed. Therefore, in most scenarios, the overhead is slightly less, but on the contrary, a little more when canceling nested transactions.

• Database defragmentation/compaction support has been implemented and the mdbx_defrag utility has been added with a set of command-line options that allow you to define key parameters and defragmentation limits.

• API Extension:

  • the MDBX_CP_OVERWRITE option has been added to the database copy function (overwriting the target file), and the mdbx_copy utility has a similar command-line option -f.
  • added the functions mdbx_cursor_bunch_delete() and mdbx_cursor_delete_range(), which perform massive deletion of adjacent elements much faster by excluding pages and branches with deleted items from a B-tree entirely.
  • added data retrieval functions with "caching" the mdbx_cache_get() and mdbx_cache_get_SingleThreaded().
  • added the mdbx_txn_refresh() function to quickly refresh the reading transaction.
  • added the mdbx_txn_checkpoint() function to commit write transaction without releasing locks.
  • added the mdbx_txn_commit_embark_read() function to commit a writing transaction and start a reading one without interfering with other changes.
  • added the mdbx_txn_amend() function to change data starting from a snapshot of the data used in a given read transaction.
  • added the mdbx_txn_rollback() function to abort and restart a transaction with the cancellation of all changes, but without releasing locks.
  • added support for cloning reading transactions using mdbx_txn_clone().
  • added support for nested read-only transactions.
  • added the mdbx_gc_info() function to get information about GC, page usage, and the ability to iterate GC content.
  • added the mdbx_env_defrag() function for explicit DB defragmentation, as well as the mdbx_defrag utility.
  • added the MDBX_opt_split_reserve option to control the fullness of tree pages when splitting them.
  • added the functions mdbx_cursor_distance(), mdbx_cursor_scroll() and mdbx_cursor_distribute() to simplify multithreaded parallel scanning.

• Support for Harmony OS (OHOS) and Haiku OS.

• Floating-point operations are no longer used both inside the library and in utilities, and linking to libm has been removed from build scripts.

• It is possible to set debugging options MDBX_DBG_ASSERT, MDBX_DBG_AUDIT and others through environment variables. However, the corresponding debugging capabilities still need to be activated during the build.

• Expanded and redesigned the composition of information generated by the function mdbx_chk_env() and the output utility mdbx_chk.

• The main libmdbx repository has been migrated from GitFlic to SourceCraft.

  For my part, I am saddened by the need to perform such manipulations, because they create significant inconvenience to users, but (unfortunately) there are sufficient reasons for this:

  • Instead of the promised internationalization, GitFlic has only a Russian-language localization with a lot of technical features that make it difficult to use machine translation systems. This made it impossible for many users to use the service and generated a number of legitimate complaints/reproaches, including from developers from China, Brazil, Korea, Iran, etc.
  • In the three years since migrating to GitFlic, several outrageous errors in the markdown editor have not been fixed, which turned the design of releases into an annoying struggle. In addition, the development roadmap has disappeared from the public space. In total, this forced me to abandon GitFlic.

• The number of page receiving/loading operations has been added to the transaction statistics, which makes it possible to quantify the amount of work with cursors and the effectiveness of various indexing and data retrieval approaches.
  The collection of relevant statistics is controlled by the additional build option MDBX_ENABLE_PGET_STAT.

• The command-line options -b number, -L megabytes, -d percent and -G geometry have been added to the mdbx_load utility, allowing you to set the size of batch inserts, limit the volume of transactions, set the desired page filling density and redefine the geometry of the database when loading data from a dump.

• Search was accelerated by using a branchless algorithm and embedding code of built-in/default comparators.

• Redesigned internal verification statements and related build options.
  At the same time, NDEBUG no longer affects checks in the main engine code, which eliminates the causes of unexpected performance drops due to the lack of a definition of NDEBUG in non-debugging builds of users.

  The checks are divided into three categories (cheap, medium, expensive), controlled by the build option MDBX_CHECKING, which takes values from -1 to 3 inclusive.
  The value of 3 corresponds to the maximum number of checks, and -1 disables both all assert() and ENSURE() checks.
  By default, MDBX_CHECKING is assumed to be equal to the MDBX_DEBUG option, which in turn defaults to 0, which corresponds to a regular (non-debugging) library build.
  This way, compatibility with the previous behavior is maintained and at the same time precise control of debugging checks is ensured.

Behavior change:

• Re-enabled/enabled on older Linux kernels, starting with version 3.16, since now there is no reason to stop working on 3.16 while supporting 4.x kernels, an...

libmdbx v0.13.12 "Аврора" (Aurora)

The supporting release of a stable branch with bug fixes.

git diff' stat: 20 files changed, 191 insertions(+), 215 deletions(-)

Appreciations:

• Erigon for sponsorship.
• Chloe Cano for fuzzing, bug reporting and fixes.
• Weixie Cui for bug fixing through pull-requests.
• Alexander Kelchin (Serebrium Company) for bug reporting and prototypes of exploits.

Important:

• libmdbx project has changed its code development and distribution model.
  To get acquainted with important changes and plans, we recommend reading the compact presentation "libmdbx: successes, obstacles, goals and roadmap", which contains important explanations in the form of embedded comments.

• At a many requests of users, the change log now is kept in English.

Fixes:

• Fixed the counting of nested trees in the mdbx_chk tool for a multi-values/dupsort hives (backport).
• Fixed swapping the shrink/growth parameters in a DB-geometry in the mdbx_load utility (backport).
• Fixed SIGSEGV in the case when all meta pages are not entirely usable (backport).
• Fixed information collection via kstat() for bootid on Solaris and related platforms (backport).
• Fixed a typo in the ST_EXPORTED processing path that broke the build on platforms where the mentioned flag is defined for fstatvfs() (backport).
• Fixed missing error handling of NtUnmapViewOfSection() and munmap() inside osal_munmap() (backport).
• Fixed SIGSEGV due to an attempt to clean/overwrite a corrupted meta page when opening the database in read-only mode (backport).
• Fixed returning a wrong/uninitialized ms_psize field when using mdbx_enumerate_tables() (backport).
• Fixed adjusting of neighbor nested cursors after a dupsort item deletion (backport).

Other:

• The dumps generated by the mdbx_dump utility no longer contains the current size of a DB and maxreaders parameter, so that the contents of a dump depend only on a payload data.
• Cutoff packages/ into the separate aside repository.

Downloads:

• 337K, libmdbx-amalgamated-0.13.12.zpaq, sha256sum: 6cac6c2269d4f14f9d693190efcba8bb365966b90616682f3de3db04a8f491f2

• 454K, libmdbx-amalgamated-0.13.12.tar.xz, sha256sum: 4795c8f09fb9111982d0f1dd8116020ff8a098da68d086f3638bd20d16da12a4

---

For more information please see ChangeLog in the stable branch.

libmdbx v0.13.11 "A7A5" (just as Stable)

The supporting release of a stable branch with bug fixes.

git diff' stat: 159 files changed, 1911 insertions(+), 642 deletions(-)

Appreciations:

• Erigon for sponsorship.
• Cosmin Apreutesei for bug reporting.
• Igor Ikonopistsev for NodeJS bindings.
• Anatoly Zhmur for reporting bugs.

Important:

• libmdbx project has changed its code development and distribution model.
  To get acquainted with important changes and plans, we recommend reading the compact presentation "libmdbx: successes, obstacles, goals and roadmap", which contains important explanations in the form of embedded comments.

• At a many requests of users, the change log and the release notes now is kept in English.

Fixes:

• Fixed using of the identifier ERROR_UNHANDLED_ERROR, which is not defined in new versions of Windows SDK.

• Fixed non-closing of DBI descriptors for tables created in nested transactions when such transactions are aborted.

• Fixed MSVC warning C4324 for buffer::silo::bin.

• Added workaround for MinGW build failures of dll-enabled C++ API.

• Fixed extra msync/fsync operations during GC reclaiming in a full database.

• Fixed suboptimal reducing/merging of histogram items in an output of mdbx_chk tool.

• Added backlog-adjustment trick to avoid extra-growth of DB instead of reclaim GC in a rare specific cases.
  In the 0.14.x version series, this mechanism has been completely updated.
  However, for version 0.13.11, this inelegant modification done in order to reduce the risk of regression.

• Minor fixed iov_callback4dirtypages() and assertion inside cursor_put() to avoid Valgrind/ASAN warnings.

• Fixed cursor(s) finalization after ones explicitly closed in a nested transactions.

Other:

• Clarification and addition of documentation.

• Considering extended ASCII as non-printable if UTF8 enabled in C++ API.

• Using hex dump of mdbx::slice instead of base58 in C++ API.

• Added link to mdbxmou bindings for NodeJS.

Downloads:

• 338K, libmdbx-amalgamated-0.13.11.zpaq, sha256sum: 669ae89c72701e8a69cf24034a22b94b19cd60587da061f84cc861e7bb49b41b

• 455K, libmdbx-amalgamated-0.13.11.tar.xz, sha256sum: 0b3c86131147c4e23164fbbda03c120fde36eb4f38896dfa9ef6abc88f75f9c3

---

For more information please see ChangeLog in the stable branch.

v0.13.10 "Gloss of Glory" (Блеск Славы)

The supporting release of a stable branch with bug fixes, in memory of Michael Alexander Gloss, American, son of Deputy Director of the CIA, who died heroically in search of justice for the bright ideals of humanity and was posthumously awarded the Order of Courage.

For more information please see ChangeLog in the stable branch.

Downloads:

• 331K, libmdbx-amalgamated-0.13.10.zpaq, sha256sum: b04d0f739b2f033f28ae5ecb3fa1332be96420bc02984bebcb9d32ed1c7ed312

• 446K, libmdbx-amalgamated-0.13.10.tar.xz, sha256sum: e6c9af085390c41d101fce0a72794c77159e1e271e41077ea0fd4270b43cc56c

---

We relocated to SourceCraft.dev

The libmdbx project has been completely relocated to the jurisdiction of the Russian Federation.

Please refer to https://libmdbx.dqdkfa.ru for documentation and https://sourcecraft.dev/dqdkfa/libmdbx for the source code (it is still open and provided with first-class free support).

Regardless of anything else, We would like to thank Erigon once again for the sponsorship.