Repository navigation
Avoid overflow in Luksan restart counters - #657
CAOShurong wants to merge 2 commits into
Conversation
Use wider private restart counters and arithmetic for LBFGS, VAR and truncated Newton algorithms. Keep the history-size conversion after clamping, and test all seven algorithms across the 999*n overflow boundary.
|
This seems reasonable, but the test allocates an awful lot of memory. |
|
The default test now checks the shared restart arithmetic at the overflow boundaries without allocating large vectors, plus four-variable solves for all seven algorithms. The large public-API cases are still available with |
Sinkleberg
left a comment
There was a problem hiding this comment.
The new static inline at luksan.h:12 breaks the C89 build. I reproduced it with GCC 14.3, and the Linux job log shows the same compiler error before the workflow continues to the MinGW build. Could you move the helper into a C file and leave a prototype in the header? The overflow boundary tests passed locally.
Fixes #632.
The
999*nrestart interval overflows a 32-bit integer at 2,149,634 variables. LBFGS, VAR1/VAR2 and all four truncated-Newton variants then returnNLOPT_FAILUREafter the first objective evaluation.Use
long longfor the private restart counter and shared interval arithmetic, and clamp the LBFGS history length before converting it toint. The public dimension API is unchanged.The default regression checks the overflow boundaries without large allocations and runs all seven algorithms on four variables. The large public-API cases remain available with
t_luksan_restart --large.Validation on Windows/MSVC: 89 C/C++ CTest cases and 53 C-only AddressSanitizer cases pass. The default regression passes 13 cases and peaks at about 5.4 MiB. The opt-in stress test passes all 27 cases with the fixed installed library and fails seven against the unmodified library. A deliberately narrowed arithmetic helper also fails the boundary checks.