Skip to content

fix: bound JSONB decimal scale to prevent excessive CPU usage - #7858

Open
wenshao wants to merge 3 commits into
mainfrom
fix/jsonb-decimal-scale-7857
Open

fix: bound JSONB decimal scale to prevent excessive CPU usage#7858
wenshao wants to merge 3 commits into
mainfrom
fix/jsonb-decimal-scale-7857

Conversation

@wenshao

@wenshao wenshao commented Sep 5, 2026

Copy link
Copy Markdown
Member

What this PR does / why we need it?

Fixes #7857. Small JSONB payloads with excessive BC_DECIMAL scales can trigger expensive power-of-ten calculations during numeric conversion, including nested decimal values and scales.

Summary of your change

  • readDecimalScale() peeks the scale-field type byte and accepts only the four writeInt32 shapes plus BC_DECIMAL (nested computed scales) and BC_TRUE/FALSE; other encodings are rejected before decoding, closing lossy-narrowing bypasses (string or int64 scale fields).
  • Validation is deferred: every BC_DECIMAL arm fully consumes scale + unscaled value before throwing, so readers left mid-record (e.g. NullOnError field readers) resume aligned and neighbouring fields are not corrupted.
  • A cumulative budget (decimalScaleSum) bounds the composed exponent of nested decimals, and a shared depth counter bounds decimal recursion at context.maxLevel, throwing JSONException("level too large : ...") instead of StackOverflowError.
  • String-derived scales are bounded at BigDecimal construction (JSONFactory.checkDecimalScale(BigDecimal)) at exactly the 10 arms whose scale escapes or whose reduction is expensive.
  • JSONWriterJSONB.writeDecimal applies the same bound, so the library never emits JSONB it refuses to read; JSONBDump shares the same check.
  • skipValue keeps only the encoding check (no magnitude bound), preserving unknown-field tolerance.
  • Also fixes the pre-existing floatValue/doubleValue truncation in the two BC_DECIMAL float/double arms.

Compatibility

defaultDecimalMaxScale (2048) is now enforced on both ends of the JSONB format: writing a BigDecimal with |scale| > 2048 throws JSONException("scale overflow : ..."), and JSONB persisted by earlier versions with such scales is rejected at read time. Worth calling out in the release notes. Residual known limitation: a type-whitelist rejection of an exotic, hand-crafted scale-field encoding still throws at the scale-field start (same mid-record rejection class as the pre-existing checkBigintLen).

Verification

  • Issue7857: 11 tests covering both reported payloads, encoding bypasses, nested composition, depth, string-derived scales, writer/dump bounds, NullOnError resync, unknown-field skipping, and accepted boundaries.
  • mvn -pl core test: 8000 tests, 0 failures. mvn -pl extension,fastjson1-compatible test: 1411 tests, 0 failures.
  • mvn validate: passed across all 15 active modules (0 Checkstyle violations).

Please indicate you've done the following:

  • Made sure tests are passing and test coverage is added if needed.
  • Made sure commit message follow the rule of Conventional Commits specification.
  • Considered the docs impact and opened a new docs issue or PR with docs changes if needed.
中文说明

目的

修复 #7857:极小的 JSONB 输入可通过 BC_DECIMAL 的超大 scale,在数值转换时触发高成本的 10 次幂计算,嵌套 decimal 值及嵌套 scale 同样受影响。

改动

  • readDecimalScale() 先窥探 scale 字段的类型字节,仅接受 writeInt32 的四种形态外加 BC_DECIMAL(嵌套计算 scale)与 BC_TRUE/FALSE;其它编码在解码前拒绝,堵住有损窄化绕过(字符串或 int64 编码的 scale 字段)。
  • 延迟校验:每个 BC_DECIMAL 分支先完整消费 scale 与 unscaled value 再抛异常,NullOnError 等中途接管读取的场景在记录边界对齐恢复,不再腐蚀相邻字段。
  • 累计预算(decimalScaleSum)约束嵌套 decimal 的合成指数;共享深度计数把 decimal 递归限制在 context.maxLevel,深嵌套抛 JSONException("level too large : ...") 而非 StackOverflowError
  • 字符串构造的 scale 在 BigDecimal 构造点(JSONFactory.checkDecimalScale(BigDecimal))限界,恰好覆盖 10 处 scale 外泄或归约昂贵的分支。
  • JSONWriterJSONB.writeDecimal 应用同一上界,库不会写出自己拒绝读取的 JSONB;JSONBDump 共享同一校验。
  • skipValue 只保留编码校验(不做量级限界),保留未知字段容错。
  • 顺带修复两个 BC_DECIMAL float/double 分支既有的 floatValue/doubleValue 截断问题。

兼容性

defaultDecimalMaxScale(2048)现在在 JSONB 格式两端同时生效:写出 |scale| > 2048 的 BigDecimal 会抛 JSONException("scale overflow : ..."),此前版本持久化的含此类 scale 的 JSONB 在读取时也会被拒绝,建议在发布说明中披露。已知残留限制:对 exotic 手工编码的 scale 字段,类型白名单仍在 scale 字段起始处拒绝(与既有 checkBigintLen 同属记录中途拒绝类别)。

验证

  • Issue7857:11 个测试,覆盖 issue 两个样本、编码绕过、嵌套合成、深度、字符串 scale、writer/dump 限界、NullOnError 对齐恢复、未知字段跳过及合法边界。
  • mvn -pl core test:8000 个测试,0 失败;mvn -pl extension,fastjson1-compatible test:1411 个测试,0 失败。
  • mvn validate:全部 15 个启用模块通过(0 Checkstyle 违规)。

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>

@wenshao wenshao left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Downgraded from Request changes to Comment: self-PR. Partially reviewed — gaps disclosed. Suggestions are inline.

Not reviewed: build-and-test beyond core/extension/fastjson1-compatible — Agent 7 scoped tests to the changed module plus the two modules that compile against it (mvn -pl core,extension,fastjson1-compatible test: 9,402 tests, 0 failures, 0 errors, 0 skipped; mvn -pl core validate: 0 Checkstyle violations with includeTestSourceDirectory=true), and did NOT run benchmark, kotlin, extension-jaxrs, extension-solon, extension-spring5, example-solon-test, example-spring-test, safemode-test, or the profile-gated test-jdk25 / benchmark_25 / extension-spring6 / codegen modules — those were grep-scanned for large-scale BigDecimal usage (setScale with 4+ digits, 4+ digit exponent literals) with zero candidates, which is a grep, not a test run.

Not reviewed: test-efficacy mutation probe — the bundled harness is npm-only and could not scope this Maven repo (test-efficacy exited 1 with ENOENT on package.json; base-tree reported available: false, 'the base tree did not build'), so no mechanical per-hunk revert or mutant sweep was produced; reported as infrastructure, never as a finding against the PR, and partially substituted by hand: verification ran real mutants for R1-10 (HALF_UP on readInt32Value0's BC_DECIMAL arm), R1-12 (floatValue/doubleValue at :3868/:4020), R1-16 (the type-byte whitelist, with and without BC_DECIMAL) and R1-18 (skipValue-before-throw), each against the PR's own compiled Issue7857 test class.

Not explored to full depth (tool budget reached): "agent reverse-audit (round 6)": did not read ObjectReaderCreatorASM.java:3029 's generated NullOnError branch body — I confirmed by grep that the ASM path carries the same feature check but…; "agent reverse-audit (round 6)": did not read fastjson2-checks.xml 's ImportOrder (:99-105) and RegexpMultiline (:10-38) rule bodies against the new test file, so the checkstyle walk above….

中文说明

⚠️ 已从请求修改降级为评论:self-PR。 仅完成部分审查,审查缺口已披露。 建议见行内评论。

未审查(原文为英文):build-and-test beyond core/extension/fastjson1-compatible — Agent 7 scoped tests to the changed module plus the two modules that compile against it (mvn -pl core,extension,fastjson1-compatible test: 9,402 tests, 0 failures, 0 errors, 0 skipped; mvn -pl core validate: 0 Checkstyle violations with includeTestSourceDirectory=true), and did NOT run benchmark, kotlin, extension-jaxrs, extension-solon, extension-spring5, example-solon-test, example-spring-test, safemode-test, or the profile-gated test-jdk25 / benchmark_25 / extension-spring6 / codegen modules — those were grep-scanned for large-scale BigDecimal usage (setScale with 4+ digits, 4+ digit exponent literals) with zero candidates, which is a grep, not a test run.

未审查(原文为英文):test-efficacy mutation probe — the bundled harness is npm-only and could not scope this Maven repo (test-efficacy exited 1 with ENOENT on package.json; base-tree reported available: false, 'the base tree did not build'), so no mechanical per-hunk revert or mutant sweep was produced; reported as infrastructure, never as a finding against the PR, and partially substituted by hand: verification ran real mutants for R1-10 (HALF_UP on readInt32Value0's BC_DECIMAL arm), R1-12 (floatValue/doubleValue at :3868/:4020), R1-16 (the type-byte whitelist, with and without BC_DECIMAL) and R1-18 (skipValue-before-throw), each against the PR's own compiled Issue7857 test class.

未探索到全部深度(达到工具调用预算):"agent reverse-audit (round 6)"did not read ObjectReaderCreatorASM.java:3029 's generated NullOnError branch body — I confirmed by grep that the ASM path carries the same feature check but…"agent reverse-audit (round 6)"did not read fastjson2-checks.xml 's ImportOrder (:99-105) and RegexpMultiline (:10-38) rule bodies against the new test file, so the checkstyle walk above…

— qwen3.8-max via Qwen Code /review (v0.23.0)

Comment thread core/src/main/java/com/alibaba/fastjson2/JSONReaderJSONB.java Outdated
Comment thread core/src/main/java/com/alibaba/fastjson2/JSONReaderJSONB.java Outdated
Comment thread core/src/main/java/com/alibaba/fastjson2/JSONReaderJSONB.java
Comment thread core/src/main/java/com/alibaba/fastjson2/JSONReaderJSONB.java
Comment thread core/src/main/java/com/alibaba/fastjson2/JSONReaderJSONB.java Outdated
Comment thread core/src/main/java/com/alibaba/fastjson2/JSONReaderJSONB.java
Comment thread core/src/main/java/com/alibaba/fastjson2/JSONReaderJSONB.java
Comment thread core/src/main/java/com/alibaba/fastjson2/JSONReaderJSONB.java
Comment thread core/src/test/java/com/alibaba/fastjson2/issues_7800/Issue7857.java Outdated
Comment thread core/src/test/java/com/alibaba/fastjson2/issues_7800/Issue7857.java Outdated
wenshao and others added 2 commits September 6, 2026 16:19
…passes (#7857)

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] JSONB 解析未对 BC_DECIMAL 的 scale 设上界,小输入可触发数十秒级 CPU 计算

1 participant