Makefile: native-tuned default build (clang + -march=native) for plain 'make' (+60% XXH3/XXH128 on AVX-512 hosts)

[ksenxx]

What

Makefile-only change: for the fully-default invocation (plain make, with no CC=, no CFLAGS=, and no runtime-dispatch build requested), the build now

1. prefers clang (fallback clang-18) over the built-in cc default when clang is installed, and
2. adds -march=native to the default CFLAGS (-O3 -march=native), so the whole library — not just the target-attribute-compiled dispatch unit — is compiled for the host ISA.

No library source line is touched, and hash values are bit-identical in every configuration.

Why

On AVX-512 hardware, the portable default (SSE2 baseline + runtime dispatch) leaves a large amount of XXH3/XXH128 throughput on the table. Measured with the official benchmark (xxhsum -b, 100KB, taskset-pinned, best of 5) on an Intel Xeon (Cascade Lake, AVX-512), Ubuntu 24.04, GCC 13.3.0 / clang 18.1.3:

Variant	default make (upstream)	default make (this PR)	delta
XXH32	6408.6 MB/s	6422.6 MB/s	+0.2%
XXH64	12815.9 MB/s	12828.2 MB/s	+0.1%
XXH3_64b	36666.0 MB/s	58773.2 MB/s	+60.3%
XXH128	36531.6 MB/s	58529.5 MB/s	+60.2%

About 44 of those 60 points come from -march=native (AVX-512 code paths for the whole build), and the rest from clang generating a tighter AVX-512 XXH3 kernel than GCC 13 on this class of CPU.

Behavior contract (what does not change)

The tuning applies only when the user has expressed no preference at all. Any explicit choice restores exact upstream behavior:

• CC=... (command line or environment): no compiler substitution, no -march=native, upstream DISPATCH default.
• CFLAGS=...: never modified; supplying it also turns the tuning off and restores the upstream DISPATCH default.
• DISPATCH=1, LIBXXH_DISPATCH=1, or the dispatch target: portable runtime-dispatch build, never receives -march=native (this intentionally overrides even an explicit NATIVE=1).
• When native tuning is off, DISPATCH still defaults to 1 on x86/x64 exactly like today. When native tuning is active it defaults to 0, since the whole binary already targets the host ISA.

Knobs:

• NATIVE=0 — never add -march=native (the right setting before make install when binaries will be copied to other, possibly older, machines).
• NATIVE=1 — request native tuning even together with an explicit CC=....

Safety guards

-march=native is skipped automatically whenever it can't or shouldn't be used:

• cross-compilation: a --target=/-target/-arch/-m32/-mx32/-m16 option anywhere in CC/CFLAGS/CPPFLAGS/MOREFLAGS, or $(CC) -dumpmachine reporting a machine different from uname -m;
• a compiler that rejects -march=native (probed with a null compile);
• a multi-word or shell-metacharacter CC value — the probes only ever execute a plain single-word CC, checked with pure make string functions (no shell involved), and probe output is sanitized through tr before make compares it.

The CC substitution is exported so recursive makes (tests/, tests/bench, tests/collisions) build with the same compiler as the top-level artifacts. One small related fix: gcc-og-test now pins CC=gcc so it keeps testing GCC when the default compiler is substituted.

Testing

• make check passes for: default, CC=gcc, NATIVE=0, DISPATCH=1, and CFLAGS=-O2 configurations.
• Differential testing against upstream confirms bit-identical hashes for every variant (one-shot and streaming) across all of the above builds.
• Non-portability risk is limited to the fully-default build, and is documented in the Makefile comment block along with the NATIVE=0 escape hatch.

I realize changing the default build's portability is a real tradeoff (distro packagers and anyone running make install for another machine must now pass NATIVE=0 or explicit CFLAGS, as they typically already do). If you'd prefer the conservative variant — keeping upstream defaults and only adding the NATIVE=1 opt-in knob plus the guards — I'm happy to rework the PR that way; the mechanism is identical and the same +60% is then one flag away.