Skip to content

build: Tweak linker flags for zstd #11111

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
cosmo0920 wants to merge 2 commits into
base: master
Choose a base branch
from
+46 −8
Open

build: Tweak linker flags for zstd #11111

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
ced9895
build: Tweak linker flags for zstd
cosmo0920 Nov 5, 2025
62645f7
packaging: distros: almalinux: Use system installed libzstd only for 10
cosmo0920 Dec 10, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 24 additions & 6 deletions CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -599,14 +599,32 @@ add_subdirectory(${FLB_PATH_LIB_MINIZ} EXCLUDE_FROM_ALL)

# Zstd (zstd)
if(FLB_PREFER_SYSTEM_LIB_ZSTD)
find_package(PkgConfig)
pkg_check_modules(LIBZSTD libzstd>=1.4.8)
find_package(ZSTD 1.4.8 QUIET)
if(ZSTD_FOUND)
set(LIBZSTD_FOUND TRUE)
set(LIBZSTD_LIBRARIES ZSTD::ZSTD)
else()
find_package(PkgConfig)
pkg_check_modules(LIBZSTD libzstd>=1.4.8)
if(LIBZSTD_FOUND)
include_directories(${LIBZSTD_INCLUDE_DIRS})
link_directories(${LIBZSTD_LIBRARY_DIRS})
add_library(ZSTD::ZSTD INTERFACE IMPORTED)
set_property(TARGET ZSTD::ZSTD PROPERTY
INTERFACE_LINK_LIBRARIES "${LIBZSTD_LIBRARIES}")
set(LIBZSTD_LIBRARIES ZSTD::ZSTD)
endif()
endif()
endif()
if(LIBZSTD_FOUND)
include_directories(${LIBZSTD_INCLUDE_DIRS})
link_directories(${LIBZSTD_LIBRARY_DIRS})
else()

if(NOT LIBZSTD_FOUND)
include(cmake/zstd.cmake)

if(NOT MSVC)
target_compile_options(libzstd_static PRIVATE -fvisibility=hidden)
endif()

set(LIBZSTD_LIBRARIES libzstd_static)
endif()
Comment on lines +620 to 628
Copy link

@coderabbitai coderabbitai bot Dec 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

# Find and examine the CMakeLists.txt file around lines 620-628
cd /repo 2>/dev/null || cd $(pwd)
find . -name "CMakeLists.txt" -type f | head -20

Repository: fluent/fluent-bit

Length of output: 769

🏁 Script executed:

# Search for CMakeLists.txt in root and look at the ZSTD configuration section
if [ -f ./CMakeLists.txt ]; then
  echo "=== Lines 600-650 of root CMakeLists.txt ==="
  sed -n '600,650p' ./CMakeLists.txt
fi

Repository: fluent/fluent-bit

Length of output: 1397

🏁 Script executed:

# Check if --exclude-libs is used anywhere in the project
rg "exclude-libs" --type cmake

Repository: fluent/fluent-bit

Length of output: 43

🏁 Script executed:

# Look at cmake/zstd.cmake to see what it does
if [ -f ./cmake/zstd.cmake ]; then
  echo "=== cmake/zstd.cmake content ==="
  cat -n ./cmake/zstd.cmake
fi

Repository: fluent/fluent-bit

Length of output: 534

🏁 Script executed:

# Search for ZSTD-related target_link_options or target_compile_options
rg "libzstd" --type cmake -A 3 -B 1 | head -100

Repository: fluent/fluent-bit

Length of output: 7220

🏁 Script executed:

# Search for target_link_options with exclude-libs or similar linker protections
rg "target_link_options|LINKER:" --type cmake -i | grep -i "zstd\|exclude\|visibility"

Repository: fluent/fluent-bit

Length of output: 43

🏁 Script executed:

# Check if fluent-bit main executable applies any linking flags to hide zstd
rg "fluent-bit-static|fluent-bit-shared" ./CMakeLists.txt -A 5 | head -50

Repository: fluent/fluent-bit

Length of output: 43

🏁 Script executed:

# Search for any hidden/version scripts that might control symbol visibility
fd -e "map\|version\|ld" --type f | grep -i "script\|version\|ld"

Repository: fluent/fluent-bit

Length of output: 43

🏁 Script executed:

# Look for any git history or comments about symbol visibility issues
rg "symbol.*hiding|visibility.*linker|exclude-libs|interposition" . --type-list | head -20

Repository: fluent/fluent-bit

Length of output: 450

🏁 Script executed:

# Check if there are any TODOs or FIXMEs related to ZSTD symbol visibility
rg "TODO|FIXME|XXX" ./CMakeLists.txt -B 2 -A 2 | grep -i "zstd\|symbol\|visibility" -A 3 -B 3

Repository: fluent/fluent-bit

Length of output: 43

🏁 Script executed:

# Look at the actual bundled zstd to understand the ZSTDLIB_VISIBLE macro behavior
head -50 ./lib/zstd-1.5.7/lib/zstd.h | grep -i "ZSTDLIB_VISIBLE\|ZSTDLIB_HIDDEN"

Repository: fluent/fluent-bit

Length of output: 590

🏁 Script executed:

# Check if zstd headers define these macros conditionally
fd "zstd\.h" --type f | xargs grep -l "ZSTDLIB_VISIBLE" | head -5

Repository: fluent/fluent-bit

Length of output: 86

🏁 Script executed:

# Look at one of the zstd headers to understand the visibility macro definitions
if [ -f ./lib/zstd-1.5.7/lib/zstd.h ]; then
  grep -A 5 -B 5 "define ZSTDLIB_VISIBLE" ./lib/zstd-1.5.7/lib/zstd.h | head -30
fi

Repository: fluent/fluent-bit

Length of output: 567

🏁 Script executed:

# Verify which zstd functions use ZSTDLIB_VISIBLE annotation
grep "ZSTDLIB_VISIBLE" ./lib/zstd-1.5.7/lib/zstd.h | head -20

Repository: fluent/fluent-bit

Length of output: 696

Symbol hiding for vendored ZSTD is incomplete—the -fvisibility=hidden flag alone does not prevent public symbol export.

The bundled zstd library (lib/zstd-1.5.7) marks its public API with ZSTDLIB_VISIBLE and ZSTDLIB_STATIC_API, which expand to __attribute__ ((visibility ("default"))) on GCC/Clang. This explicit visibility attribute overrides the compile-time -fvisibility=hidden flag applied at line 624, meaning zstd's symbols are still exported in the final binary and remain vulnerable to interposition by a system-installed libzstd at runtime.

Additionally:

  • No linker-level protection: The --exclude-libs linker flag is not applied, which would prevent exported symbols from being added to the executable's dynamic symbol table.
  • No MSVC/Windows handling: On Windows, ZSTDLIB_API uses __declspec(dllexport), and there is no corresponding mechanism to prevent symbol visibility in the static library context.

Recommendation:
Apply linker-level symbol hiding: target_link_options(fluent-bit-static PRIVATE "LINKER:--exclude-libs,libzstd_static") after the zstd target is linked, or alternatively define ZSTDLIB_VISIBILITY to an empty string during zstd's build to override the default visibility macros.

🤖 Prompt for AI Agents 
In CMakeLists.txt around lines 620-628, setting -fvisibility=hidden for the
vendored libzstd is insufficient because zstd explicitly marks public APIs with
visibility attributes (and uses __declspec(dllexport) on MSVC), so symbols still
get exported; fix by applying linker-level hiding and overriding zstd's
visibility macros: after creating/linking the libzstd_static target add a
target_link_options(fluent-bit-static PRIVATE
"LINKER:--exclude-libs,libzstd_static") so the linker excludes its symbols from
the dynamic symbol table, and for the zstd build add
target_compile_definitions(libzstd_static PRIVATE ZSTDLIB_VISIBILITY=
ZSTDLIB_STATIC_API=) (or equivalent -D defines) to undefine/empty the visibility
macros on all platforms (and add an MSVC-specific define or pragma if needed) so
the source-level attributes don’t force default export.


# ring buffer library
Expand Down
24 changes: 22 additions & 2 deletions packaging/distros/almalinux/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,9 @@ RUN yum -y update && \
cyrus-sasl-lib cyrus-sasl-devel openssl openssl-libs openssl-devel libyaml-devel pkgconf-pkg-config && \
yum clean all

ARG FLB_PREFER_SYSTEM_LIB_ZSTD=Off
ENV FLB_PREFER_SYSTEM_LIB_ZSTD=$FLB_PREFER_SYSTEM_LIB_ZSTD

# almalinux/8.arm64v8 base image
# hadolint ignore=DL3029
FROM --platform=arm64 almalinux:8 AS almalinux-8.arm64v8-base
Expand All @@ -40,6 +43,8 @@ RUN yum -y update && \
cyrus-sasl-lib cyrus-sasl-devel openssl openssl-libs openssl-devel libyaml-devel pkgconf-pkg-config && \
yum clean all

ARG FLB_PREFER_SYSTEM_LIB_ZSTD=Off
ENV FLB_PREFER_SYSTEM_LIB_ZSTD=$FLB_PREFER_SYSTEM_LIB_ZSTD
# Need larger page size
ARG FLB_JEMALLOC_OPTIONS="--with-lg-page=16 --with-lg-quantum=3"
ENV FLB_JEMALLOC_OPTIONS=$FLB_JEMALLOC_OPTIONS
Expand All @@ -57,6 +62,9 @@ RUN yum -y update && \
cyrus-sasl-lib cyrus-sasl-devel openssl openssl-libs openssl-devel libyaml-devel pkgconf-pkg-config && \
yum clean all

ARG FLB_PREFER_SYSTEM_LIB_ZSTD=Off
ENV FLB_PREFER_SYSTEM_LIB_ZSTD=$FLB_PREFER_SYSTEM_LIB_ZSTD

# almalinux/8.arm64v8 base image
# hadolint ignore=DL3029
FROM --platform=arm64 almalinux:9 AS almalinux-9.arm64v8-base
Expand All @@ -74,6 +82,8 @@ RUN yum -y update && \
cyrus-sasl-lib cyrus-sasl-devel openssl openssl-libs openssl-devel libyaml-devel pkgconf-pkg-config && \
yum clean all

ARG FLB_PREFER_SYSTEM_LIB_ZSTD=Off
ENV FLB_PREFER_SYSTEM_LIB_ZSTD=$FLB_PREFER_SYSTEM_LIB_ZSTD
# Need larger page size
ARG FLB_JEMALLOC_OPTIONS="--with-lg-page=16 --with-lg-quantum=3"
ENV FLB_JEMALLOC_OPTIONS=$FLB_JEMALLOC_OPTIONS
Expand All @@ -88,9 +98,14 @@ RUN yum -y update && \
yum install -y --allowerasing rpm-build curl ca-certificates gcc gcc-c++ cmake make bash \
wget unzip systemd-devel wget flex bison \
postgresql-libs postgresql-devel postgresql-server postgresql \
cyrus-sasl-lib cyrus-sasl-devel openssl openssl-libs openssl-devel libyaml-devel pkgconf-pkg-config && \
cyrus-sasl-lib cyrus-sasl-devel openssl openssl-libs openssl-devel libyaml-devel pkgconf-pkg-config \
libzstd-devel && \
yum clean all

# Use system installed libzstd library.
ARG FLB_PREFER_SYSTEM_LIB_ZSTD=On
ENV FLB_PREFER_SYSTEM_LIB_ZSTD=$FLB_PREFER_SYSTEM_LIB_ZSTD

# almalinux/8.arm64v8 base image
# hadolint ignore=DL3029
FROM --platform=arm64 almalinux:10 AS almalinux-10.arm64v8-base
Expand All @@ -105,9 +120,13 @@ RUN yum -y update && \
yum install -y --allowerasing rpm-build curl ca-certificates gcc gcc-c++ cmake make bash \
wget unzip systemd-devel wget flex bison \
postgresql-libs postgresql-devel postgresql-server postgresql \
cyrus-sasl-lib cyrus-sasl-devel openssl openssl-libs openssl-devel libyaml-devel pkgconf-pkg-config && \
cyrus-sasl-lib cyrus-sasl-devel openssl openssl-libs openssl-devel libyaml-devel pkgconf-pkg-config \
libzstd-devel && \
yum clean all

# Use system installed libzstd library.
ARG FLB_PREFER_SYSTEM_LIB_ZSTD=On
ENV FLB_PREFER_SYSTEM_LIB_ZSTD=$FLB_PREFER_SYSTEM_LIB_ZSTD
# Need larger page size
ARG FLB_JEMALLOC_OPTIONS="--with-lg-page=16 --with-lg-quantum=3"
ENV FLB_JEMALLOC_OPTIONS=$FLB_JEMALLOC_OPTIONS
Expand Down Expand Up @@ -155,6 +174,7 @@ RUN cmake -DCMAKE_INSTALL_PREFIX="$CMAKE_INSTALL_PREFIX" \
-DFLB_JEMALLOC="${FLB_JEMALLOC}" \
-DFLB_CHUNK_TRACE="${FLB_CHUNK_TRACE}" \
-DFLB_UNICODE_ENCODER="${FLB_UNICODE_ENCODER}" \
-DFLB_PREFER_SYSTEM_LIB_ZSTD="${FLB_PREFER_SYSTEM_LIB_ZSTD}" \
../

VOLUME [ "/output" ]
Expand Down
Loading