fix: bound JSONB decimal scale to prevent excessive CPU usage - #7858
fix: bound JSONB decimal scale to prevent excessive CPU usage#7858wenshao wants to merge 3 commits into
Conversation
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
wenshao
left a comment
There was a problem hiding this comment.
Not reviewed: build-and-test beyond core/extension/fastjson1-compatible — Agent 7 scoped tests to the changed module plus the two modules that compile against it (mvn -pl core,extension,fastjson1-compatible test: 9,402 tests, 0 failures, 0 errors, 0 skipped; mvn -pl core validate: 0 Checkstyle violations with includeTestSourceDirectory=true), and did NOT run benchmark, kotlin, extension-jaxrs, extension-solon, extension-spring5, example-solon-test, example-spring-test, safemode-test, or the profile-gated test-jdk25 / benchmark_25 / extension-spring6 / codegen modules — those were grep-scanned for large-scale BigDecimal usage (setScale with 4+ digits, 4+ digit exponent literals) with zero candidates, which is a grep, not a test run.
Not reviewed: test-efficacy mutation probe — the bundled harness is npm-only and could not scope this Maven repo (test-efficacy exited 1 with ENOENT on package.json; base-tree reported available: false, 'the base tree did not build'), so no mechanical per-hunk revert or mutant sweep was produced; reported as infrastructure, never as a finding against the PR, and partially substituted by hand: verification ran real mutants for R1-10 (HALF_UP on readInt32Value0's BC_DECIMAL arm), R1-12 (floatValue/doubleValue at :3868/:4020), R1-16 (the type-byte whitelist, with and without BC_DECIMAL) and R1-18 (skipValue-before-throw), each against the PR's own compiled Issue7857 test class.
Not explored to full depth (tool budget reached): "agent reverse-audit (round 6)": did not read ObjectReaderCreatorASM.java:3029 's generated NullOnError branch body — I confirmed by grep that the ASM path carries the same feature check but…; "agent reverse-audit (round 6)": did not read fastjson2-checks.xml 's ImportOrder (:99-105) and RegexpMultiline (:10-38) rule bodies against the new test file, so the checkstyle walk above….
中文说明
未审查(原文为英文):build-and-test beyond core/extension/fastjson1-compatible — Agent 7 scoped tests to the changed module plus the two modules that compile against it (mvn -pl core,extension,fastjson1-compatible test: 9,402 tests, 0 failures, 0 errors, 0 skipped; mvn -pl core validate: 0 Checkstyle violations with includeTestSourceDirectory=true), and did NOT run benchmark, kotlin, extension-jaxrs, extension-solon, extension-spring5, example-solon-test, example-spring-test, safemode-test, or the profile-gated test-jdk25 / benchmark_25 / extension-spring6 / codegen modules — those were grep-scanned for large-scale BigDecimal usage (setScale with 4+ digits, 4+ digit exponent literals) with zero candidates, which is a grep, not a test run.
未审查(原文为英文):test-efficacy mutation probe — the bundled harness is npm-only and could not scope this Maven repo (test-efficacy exited 1 with ENOENT on package.json; base-tree reported available: false, 'the base tree did not build'), so no mechanical per-hunk revert or mutant sweep was produced; reported as infrastructure, never as a finding against the PR, and partially substituted by hand: verification ran real mutants for R1-10 (HALF_UP on readInt32Value0's BC_DECIMAL arm), R1-12 (floatValue/doubleValue at :3868/:4020), R1-16 (the type-byte whitelist, with and without BC_DECIMAL) and R1-18 (skipValue-before-throw), each against the PR's own compiled Issue7857 test class.
未探索到全部深度(达到工具调用预算):"agent reverse-audit (round 6)":did not read ObjectReaderCreatorASM.java:3029 's generated NullOnError branch body — I confirmed by grep that the ASM path carries the same feature check but…;"agent reverse-audit (round 6)":did not read fastjson2-checks.xml 's ImportOrder (:99-105) and RegexpMultiline (:10-38) rule bodies against the new test file, so the checkstyle walk above…。
— qwen3.8-max via Qwen Code /review (v0.23.0)
…passes (#7857) Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
What this PR does / why we need it?
Fixes #7857. Small JSONB payloads with excessive BC_DECIMAL scales can trigger expensive power-of-ten calculations during numeric conversion, including nested decimal values and scales.
Summary of your change
readDecimalScale()peeks the scale-field type byte and accepts only the fourwriteInt32shapes plusBC_DECIMAL(nested computed scales) andBC_TRUE/FALSE; other encodings are rejected before decoding, closing lossy-narrowing bypasses (string or int64 scale fields).BC_DECIMALarm fully consumes scale + unscaled value before throwing, so readers left mid-record (e.g.NullOnErrorfield readers) resume aligned and neighbouring fields are not corrupted.decimalScaleSum) bounds the composed exponent of nested decimals, and a shared depth counter bounds decimal recursion atcontext.maxLevel, throwingJSONException("level too large : ...")instead ofStackOverflowError.BigDecimalconstruction (JSONFactory.checkDecimalScale(BigDecimal)) at exactly the 10 arms whose scale escapes or whose reduction is expensive.JSONWriterJSONB.writeDecimalapplies the same bound, so the library never emits JSONB it refuses to read;JSONBDumpshares the same check.skipValuekeeps only the encoding check (no magnitude bound), preserving unknown-field tolerance.floatValue/doubleValuetruncation in the twoBC_DECIMALfloat/double arms.Compatibility
defaultDecimalMaxScale(2048) is now enforced on both ends of the JSONB format: writing aBigDecimalwith |scale| > 2048 throwsJSONException("scale overflow : ..."), and JSONB persisted by earlier versions with such scales is rejected at read time. Worth calling out in the release notes. Residual known limitation: a type-whitelist rejection of an exotic, hand-crafted scale-field encoding still throws at the scale-field start (same mid-record rejection class as the pre-existingcheckBigintLen).Verification
Issue7857: 11 tests covering both reported payloads, encoding bypasses, nested composition, depth, string-derived scales, writer/dump bounds, NullOnError resync, unknown-field skipping, and accepted boundaries.mvn -pl core test: 8000 tests, 0 failures.mvn -pl extension,fastjson1-compatible test: 1411 tests, 0 failures.mvn validate: passed across all 15 active modules (0 Checkstyle violations).Please indicate you've done the following:
中文说明
目的
修复 #7857:极小的 JSONB 输入可通过 BC_DECIMAL 的超大 scale,在数值转换时触发高成本的 10 次幂计算,嵌套 decimal 值及嵌套 scale 同样受影响。
改动
readDecimalScale()先窥探 scale 字段的类型字节,仅接受writeInt32的四种形态外加BC_DECIMAL(嵌套计算 scale)与BC_TRUE/FALSE;其它编码在解码前拒绝,堵住有损窄化绕过(字符串或 int64 编码的 scale 字段)。BC_DECIMAL分支先完整消费 scale 与 unscaled value 再抛异常,NullOnError等中途接管读取的场景在记录边界对齐恢复,不再腐蚀相邻字段。decimalScaleSum)约束嵌套 decimal 的合成指数;共享深度计数把 decimal 递归限制在context.maxLevel,深嵌套抛JSONException("level too large : ...")而非StackOverflowError。BigDecimal构造点(JSONFactory.checkDecimalScale(BigDecimal))限界,恰好覆盖 10 处 scale 外泄或归约昂贵的分支。JSONWriterJSONB.writeDecimal应用同一上界,库不会写出自己拒绝读取的 JSONB;JSONBDump共享同一校验。skipValue只保留编码校验(不做量级限界),保留未知字段容错。BC_DECIMALfloat/double 分支既有的floatValue/doubleValue截断问题。兼容性
defaultDecimalMaxScale(2048)现在在 JSONB 格式两端同时生效:写出 |scale| > 2048 的BigDecimal会抛JSONException("scale overflow : ..."),此前版本持久化的含此类 scale 的 JSONB 在读取时也会被拒绝,建议在发布说明中披露。已知残留限制:对 exotic 手工编码的 scale 字段,类型白名单仍在 scale 字段起始处拒绝(与既有checkBigintLen同属记录中途拒绝类别)。验证
Issue7857:11 个测试,覆盖 issue 两个样本、编码绕过、嵌套合成、深度、字符串 scale、writer/dump 限界、NullOnError 对齐恢复、未知字段跳过及合法边界。mvn -pl core test:8000 个测试,0 失败;mvn -pl extension,fastjson1-compatible test:1411 个测试,0 失败。mvn validate:全部 15 个启用模块通过(0 Checkstyle 违规)。