Hey folks - hit this one on xtdb/xtdb#6038. AI summary below (which I've read through), let me know if anything's unclear 🙂
Cheers,
James
Version: 1.10.0. The same lexer code is present on v1.12.0-RC.
longOrNull is documented to return "null if current element is not a valid representation of number", and it does so for Long.MAX_VALUE + 1. But an integer large enough to wrap past 2^64 comes back as a small, entirely different number instead of null.
Reproducer
val p = Json.parseToJsonElement("18446744073709551623").jsonPrimitive // 2^64 + 7
println(p.longOrNull) // 7
println(p.intOrNull) // 7
Expected: null from both, as for 9223372036854775808.
Observed across the boundary:
| literal |
value |
longOrNull |
intOrNull |
9223372036854775807 |
2^63-1 |
9223372036854775807 |
null |
9223372036854775808 |
2^63 |
null |
null |
18446744073709551615 |
2^64-1 |
null |
null |
18446744073709551616 |
2^64 |
0 |
0 |
18446744073709551623 |
2^64+7 |
7 |
7 |
-18446744073709551623 |
-(2^64+7) |
-7 |
-7 |
36893488147419103232 |
2^65 |
0 |
0 |
Cause
longOrNull resolves through parseLongImpl to AbstractJsonLexer.consumeNumericLiteral, which accumulates the digits negatively and guards overflow in two places. In the digit loop:
accumulator = accumulator * 10 - digit
if (accumulator > 0) fail("Numeric value overflow")
and on the way out:
return when {
isNegative -> accumulator
accumulator != Long.MIN_VALUE -> -accumulator
else -> fail("Numeric value overflow")
}
The first catches a wrap that crosses into positive, which is what rejects 18446744073709551615 (it accumulates to 1). The second — added in #1367, for 9223372036854775808 reading as -9223372036854775808 — catches a positive literal that accumulated to exactly Long.MIN_VALUE.
Neither catches a wrap that lands on any other negative value. 18446744073709551623 accumulates to -7, stays negative, and is negated on return to 7.
Scope
long / longOrNull and int / intOrNull. The int accessors range-check after parseLongImpl, so a wrapped value that falls inside Int range passes the check.
StreamingJsonDecoder.decodeLong() and decodeInt() call the same lexer method, so a @Serializable class with a Long field silently receives the wrapped value when decoded from JSON.
Note
#1300 fixed the sibling case for narrow types decoded from a JsonElement; this is the Long path in the lexer itself.
Hey folks - hit this one on xtdb/xtdb#6038. AI summary below (which I've read through), let me know if anything's unclear 🙂
Cheers,
James
Version: 1.10.0. The same lexer code is present on
v1.12.0-RC.longOrNullis documented to return "null if current element is not a valid representation of number", and it does so forLong.MAX_VALUE + 1. But an integer large enough to wrap past 2^64 comes back as a small, entirely different number instead of null.Reproducer
Expected:
nullfrom both, as for9223372036854775808.Observed across the boundary:
longOrNullintOrNull92233720368547758079223372036854775807null9223372036854775808nullnull18446744073709551615nullnull18446744073709551616001844674407370955162377-18446744073709551623-7-73689348814741910323200Cause
longOrNullresolves throughparseLongImpltoAbstractJsonLexer.consumeNumericLiteral, which accumulates the digits negatively and guards overflow in two places. In the digit loop:and on the way out:
The first catches a wrap that crosses into positive, which is what rejects
18446744073709551615(it accumulates to1). The second — added in #1367, for9223372036854775808reading as-9223372036854775808— catches a positive literal that accumulated to exactlyLong.MIN_VALUE.Neither catches a wrap that lands on any other negative value.
18446744073709551623accumulates to-7, stays negative, and is negated on return to7.Scope
long/longOrNullandint/intOrNull. Theintaccessors range-check afterparseLongImpl, so a wrapped value that falls insideIntrange passes the check.StreamingJsonDecoder.decodeLong()anddecodeInt()call the same lexer method, so a@Serializableclass with aLongfield silently receives the wrapped value when decoded from JSON.Note
#1300 fixed the sibling case for narrow types decoded from a
JsonElement; this is theLongpath in the lexer itself.