Skip to content

[BUG] Expression evaluation differs from Neo4j, and two evaluators implement it: missing Neo4j 5 functions, list predicates over null, FLOAT to STRING formatting, INTEGER vs FLOAT above 2^53, division / temporal / nested errors swallowed or rolled over (family) #698

Description

@mm0nst3r

Bug Description

Several Neo4j 5 built-in functions, and the trim(… FROM …) form, are rejected with SyntaxError "unknown function" (or a parse error). Neo4j 5.26 returns a value for each of them.

Consequence

Severity: valid queries fail. Queries written for Neo4j 5 fail as soon as they use one of these functions. Examples: char_length for string lengths, toIntegerList to convert imported strings, upper / btrim, valueType in type checks, nullIf. radians and isNaN are also used in numeric code.

Steps to Reproduce / Expected Behavior / Actual Behavior

Neo4j Python driver 5.28.1, Bolt auto-commit, empty database:

statement Neo4j 5.26.30 (pinned) NornicDB main 8205fc5
RETURN radians(180) 3.141592653589793 SyntaxError: unknown function: radians
RETURN isNaN(0.0/0.0) true unknown function: isNaN
RETURN char_length('abc') 3 unknown function: char_length
RETURN character_length('abc') 3 unknown function: character_length
RETURN upper('a') 'A' unknown function: upper (lower works)
RETURN btrim(' a ') 'a' unknown function: btrim
RETURN ltrim('xxa', 'x') 'a' SyntaxError: could not parse RETURN expression (the one-argument form works)
RETURN normalize('a') 'a' unknown function: normalize
RETURN toIntegerList(['1','2']) [1, 2] unknown function: toIntegerList
RETURN toFloatList(['1.5']) [1.5] unknown function: toFloatList
RETURN toStringList([1]) ['1'] unknown function: toStringList
RETURN toBooleanList(['true']) [true] unknown function: toBooleanList
RETURN trim(BOTH 'x' FROM 'xax') 'a' SyntaxError: an expression is followed by …
RETURN trim(LEADING 'x' FROM 'xxa') 'a' same
RETURN valueType(1) 'INTEGER NOT NULL' unknown function: valueType
RETURN nullIf(1, 1) null unknown function: nullIf

lower, one-argument trim and toIntegerOrNull work and match Neo4j.

Environment

  • NornicDB Version: main at 8205fc5
  • Reference: neo4j:5.26.30-community (pinned in scripts/cypher-tck/run-differential.sh), neo4j Python driver 5.28.1
  • Build: -tags "noui,nolocalllm"

Checklist

  • I have searched existing issues to ensure this is not a duplicate
  • I have tested this with the latest version of NornicDB
  • I have included all relevant information (environment, query, error messages)

🤖 Generated with Claude Code

Consolidated from #736: [BUG] List predicates all() / any() / none() / single() over a null list return false / true instead of null; a literal null list is rejected

#736 is merged into this issue (same family: value semantics of the expression evaluators (the string evaluator evaluateExpressionWithContext* and the row evaluator evaluateRow*); the failing statements of every section run through them (coverage per failing statement on main 2afc84f)). Its full text (and its comments) follows; the fix for this issue must cover it.

Bug Description

The list predicates all(), any(), none() and single() over a null list return a boolean instead of null, and with a literal null list the statement is rejected.

Steps to Reproduce

Main 2afc84f over Bolt (auto-commit) against the pinned neo4j:5.26.30-community@sha256:3388e05e…:

statement Neo4j 5.26.30 NornicDB main
WITH null AS m RETURN all(x IN m WHERE x > 0) AS v null false
WITH null AS m RETURN any(x IN m WHERE x > 0) AS v null false
WITH null AS m RETURN none(x IN m WHERE x > 0) AS v null true
WITH null AS m RETURN single(x IN m WHERE x > 0) AS v null false
RETURN all(x IN null WHERE x > 0) AS v null SyntaxError: could not …
RETURN none(x IN null WHERE true) AS v null SyntaxError: could not …
UNWIND [1, 2] AS i WITH i, CASE WHEN i = 1 THEN null ELSE [1] END AS m WHERE any(x IN m WHERE x > 0) RETURN i 2 2 (same)

As a WHERE filter the result is the same, because null and false both drop the row. The difference shows wherever the value itself is used: RETURN, SET, comparisons such as none(…) = true, coalesce, and CASE.

Expected Behavior

A list predicate over a null list is null, as in Neo4j and the Cypher manual.

Consequence

Severity: wrong results returned without an error. An optional list (a missing property, an OPTIONAL MATCH collect) makes none(…) report true and all(…) false, and those values get stored by SET or returned as if they were checked. A literal null list is rejected outright.

Environment

  • NornicDB Version: main at 2afc84f
  • Reference: neo4j:5.26.30-community@sha256:3388e05e…
  • Build: golang:1.27.1-bookworm, -tags "noui nolocalllm"

🤖 Generated with Claude Code

----- (comment)
PR: #720 (commit c5b426c).

🤖 Generated with Claude Code

Consolidated from #725: [BUG] FLOAT to STRING uses Go formatting: toString(1.0) = '1', 'v=' + 1.0 = 'v=1', 1e16 → '1e+16', Infinity → '+Inf', -0.0 → '0'

#725 is merged into this issue (same family: value semantics of the expression evaluators (the string evaluator evaluateExpressionWithContext* and the row evaluator evaluateRow*); the failing statements of every section run through them (coverage per failing statement on main 2afc84f)). Its full text (and its comments) follows; the fix for this issue must cover it.

Bug Description

Converting a FLOAT to a STRING uses Go's formatting instead of Neo4j's (Java's). Whole numbers lose their .0, exponents use Go's e+16 style, infinities are +Inf / -Inf, and -0.0 becomes 0. This affects toString(), toStringOrNull(), and string concatenation ('v=' + 1.0).

Consequence

Different stored and returned text, silently. Anything that builds strings from numbers produces different values than on Neo4j: keys, labels, messages, CSV-like exports, and properties such as n.label = 'v' + n.weight. So string comparisons, joins on those keys, and any client that parses the text break when moving between Neo4j and NornicDB. toString(1.0) = '1.0' is false on NornicDB. Stored values created this way differ permanently.

Steps to Reproduce

Main 2afc84f against neo4j:5.26.30-community, Bolt auto-commit:

statement Neo4j 5.26.30 NornicDB
RETURN toString(1.0) '1.0' '1'
WITH 3.0 AS f RETURN toString(f) '3.0' '3'
RETURN 'v=' + 1.0 / RETURN 1.0 + '' 'v=1.0' / '1.0' 'v=1' / '1'
RETURN toStringOrNull(1.0) '1.0' '1'
RETURN toString(1.5e20) '1.5E20' '1.5e+20'
RETURN toString(1e16) '1.0E16' '1e+16'
RETURN toString(123456789.0) '1.23456789E8' '1.23456789e+08'
RETURN toString(1e-7) '1.0E-7' '1e-07'
RETURN toString(1.0/0.0) / toString(-1.0/0.0) 'Infinity' / '-Infinity' '+Inf' / '-Inf'
RETURN toString(-0.0) '-0.0' '0'
RETURN toString(0.1), toString(2.5), toString(0.0/0.0) '0.1', '2.5', 'NaN' same

Expected Behavior

FLOAT to STRING follows Java's Double.toString: at least one decimal digit (1.0), scientific notation below 10⁻³ and at 10⁷ or above with the E form (1.0E16, 1.23456789E8, 1.0E-7), Infinity / -Infinity / NaN, and -0.0.

Additional Context

The conversion falls back to fmt.Sprint for floats (formatCypherValueString). Every string conversion of a number (toString, concatenation, and any other place that renders a float as text) should go through one Neo4j-compatible formatter. #668 (whole-valued FLOATs sent as JSON integers over HTTP) is the same "a float rendered like an integer" problem on the HTTP encoder.

Environment

  • NornicDB Version: main at 2afc84f
  • Reference: neo4j:5.26.30-community, neo4j Python driver 5.28.1
  • Build: -tags "noui,nolocalllm", Linux container on Docker Desktop / WSL2

Checklist

  • I have searched existing issues to ensure this is not a duplicate
  • I have tested this with the latest version of NornicDB
  • I have included all relevant information (environment, query, error messages)

🤖 Generated with Claude Code

----- (comment)
Convergence note (main 2afc84f): float-to-text formatting is done ad hoc in many places in pkg/cypher, each with its own verb:

  • strconv.FormatFloat(…, 'g', -1, …): call.go:3518/3520, clauses.go:2003/2005, executor_subqueries.go:1185/1188, knowledgepolicy_procedures.go:197, parameters.go:575;
  • fmt.Sprintf("%g"): executor_subqueries.go:371/377/387/389;
  • fmt.Sprintf("%v"): call.go:3522, call_fulltext.go:258, executor_subqueries.go:402, parameters.go:570;
  • FormatFloat(…, 'f', …): executor_match_vector_cosine_fastpath.go:1509;
  • formatCypherValueString (temporal_accessors.go:11), which handles the temporal types and falls back to generic formatting for the rest.

Some of these build internal keys or literals, not user-visible text, but every user-visible FLOAT → STRING should go through one Java-compatible formatter, and the internal ones should say they're internal. Otherwise each copy keeps its own rule.

🤖 Generated with Claude Code

Consolidated from #540: [BUG] Integers above 2^53 are compared and sorted as floats: 9007199254740993 > 9007199254740992 is false, ORDER BY returns them in the wrong order

#540 is merged into this issue (same family: value semantics of the expression evaluators (the string evaluator evaluateExpressionWithContext* and the row evaluator evaluateRow*); the failing statements of every section run through them (coverage per failing statement on main 2afc84f)). Its full text (and its comments) follows; the fix for this issue must cover it.

Bug Description

Cypher integers are 64-bit. NornicDB compares them with <, >, <=, >= and sorts them in ORDER BY after converting them to 64-bit floats. Above 2^53 (9007199254740992) neighbouring integers become the same float:

  • 9007199254740993 > 9007199254740992 is false;

  • ORDER BY leaves such values in input order, ascending or descending, including WITH … ORDER BY … collect().

Equality (=) and max() are correct.

Consequence

Severity: wrong results. 64-bit identifiers are routinely above 2^53: Snowflake/Twitter-style IDs, Discord IDs, hashes stored as integers, nanosecond timestamps. Range filters (WHERE n.id > $cursor) can skip or repeat rows, and keyset pagination and "latest first" sorting return the wrong order, without any error. Equality still works, so lookups by ID look fine while ordering is broken.

Steps to Reproduce

Fresh database; driver auto-commit and managed explicit transaction:

RETURN 9007199254740993 > 9007199254740992 AS gt, 9007199254740993 = 9007199254740992 AS eq

UNWIND [9007199254740993, 9007199254740992] AS x RETURN x ORDER BY x

UNWIND [9007199254740992, 9007199254740993] AS x RETURN x ORDER BY x DESC

UNWIND [9007199254740993, 9007199254740992] AS x WITH x ORDER BY x RETURN collect(x) AS l

UNWIND [9007199254740993, 9007199254740992] AS x RETURN max(x) AS m

Expected Behavior

The Neo4j 5.26.30 column below.

Actual Behavior

| # | statement | Neo4j 5.26.30 | NornicDB auto-commit | NornicDB explicit transaction |

| --- | --- | --- | --- | --- |

| 1 | RETURN 9007199254740993 > 9007199254740992 AS gt, 9007199254740993 = 9007199254740992 AS eq | [{eq: false, gt: true}] | [{eq: false, gt: false}] | [{eq: false, gt: false}] |

| 2 | UNWIND [9007199254740993, 9007199254740992] AS x RETURN x ORDER BY x | [{x: 9007199254740992}, {x: 9007199254740993}] | [{x: 9007199254740993}, {x: 9007199254740992}] | [{x: 9007199254740993}, {x: 9007199254740992}] |

| 3 | UNWIND [9007199254740992, 9007199254740993] AS x RETURN x ORDER BY x DESC | [{x: 9007199254740993}, {x: 9007199254740992}] | [{x: 9007199254740992}, {x: 9007199254740993}] | [{x: 9007199254740992}, {x: 9007199254740993}] |

| 4 | UNWIND [9007199254740993, 9007199254740992] AS x WITH x ORDER BY x RETURN collect(x) AS l | [{l: [9007199254740992, 9007199254740993]}] | [{l: [9007199254740993, 9007199254740992]}] | [{l: [9007199254740993, 9007199254740992]}] |

| 5 | UNWIND [9007199254740993, 9007199254740992] AS x RETURN max(x) AS m | [{m: 9007199254740993}] | [{m: 9007199254740993}] | [{m: 9007199254740993}] |

Additional Context

Code-path divergence between equality and ordering on numbers in pkg/cypher:

  • numeric equality (value_equality.go, around line 85) compares integers exactly before falling back to a float comparison;

  • the comparison operators (row_expression.go, around lines 701 and 815, via strictNumericValue) and the ORDER BY comparator (executor_subqueries.go, around lines 3710 and 3775, via cypherSortNumber) always widen both sides to float64.

strictNumericValue and cypherSortNumber are separate copies of the same type switch.

Environment

  • NornicDB Version: main at de54a14 (tested on a build of 38eb1b9; the two commits after it change only _test.go files)

  • Reference: neo4j:5.26.30-community@sha256:3388e05e… (the image pinned in scripts/cypher-tck/run-differential.sh), same statements, same driver

  • OS: Linux container (debian:bookworm-slim) on Docker Desktop / WSL2

  • Architecture: AMD64

  • Docker Image: built from source, -tags "noui,nolocalllm", embeddings disabled

  • Driver: neo4j Python driver 5.28.1, Bolt

Checklist

  • I have searched existing issues to ensure this is not a duplicate

  • I have tested this with the latest version of NornicDB

  • I have included all relevant information (environment, query, error messages)

🤖 Generated with Claude Code

----- (comment)
Fix in #550

----- (comment)
Verified fixed in e7b75bc. Checked on a build of main ad82416 against the pinned neo4j:5.26.30-community@sha256:3388e05e…, fresh database before each statement, driver auto-commit and managed explicit transaction, comparing rows (and the stored graph for writes).

All five statements from this issue now match Neo4j in both modes: 9007199254740993 > 9007199254740992 → true; ORDER BY ascending, descending and WITH … ORDER BY … collect() return the exact order; max() unchanged.

Additional cases, all identical to Neo4j:

statement result
9007199254740993 >= 9007199254740993, 9007199254740993 < 9007199254740994 true, true
9007199254740993 > 9007199254740992.0 (integer vs float) false (float comparison, as in Neo4j)
min() over [9007199254740993, 9007199254740992, 1.5, -9007199254740993] -9007199254740993
ORDER BY x DESC over [9007199254740993, 9007199254740992, 1.5] [9007199254740993, 9007199254740992, 1.5]

----- (comment)
Reopening: with a property index on the sorted key, an explicit transaction still orders integers above 2^53 as floats. Checked on main 4c56c2e against the pinned Neo4j 5.26.30.

Fresh database before each run, with CREATE INDEX t_v FOR (n:T) ON (n.v) and CREATE (:T {id: 1, v: 9007199254740993})-[:R {w: 1}]->(:T {id: 2, v: 9007199254740992}), (:U {id: 3}); statement MATCH (n:T) RETURN n.id AS id ORDER BY n.v LIMIT 1, 6 separate runs:

route Neo4j 5.26.30 NornicDB main 4c56c2e, 6 runs
driver auto-commit [{id: 2}] [{id: 2}] ×6
HTTP /tx/commit [{id: 2}] [{id: 2}] ×6
managed explicit transaction [{id: 2}] [{id: 1}] ×4, [{id: 2}] ×2

The two values compare equal as floats (both become 9007199254740992.0), so the explicit-transaction route with the index returns either row. Without the index, all three routes return [{id: 2}] in 5 of 5 runs, and ORDER BY n.v DESC LIMIT 1 is correct as well. The same flip-flop was on the #579 branch and on PRs #630 / #633 (so it isn't introduced by them). It was missed when this was closed because it isn't deterministic.

Consequence

Severity: wrong results (intermittent, explicit transactions with an index). Top-N by a 64-bit ID or timestamp in nanoseconds returns the wrong row some of the time inside managed transactions.

🤖 Generated with Claude Code

----- (comment)

Section: INTEGER = FLOAT above 2^53 compares as floats (Neo4j compares exactly)

Main 2afc84f and #737's head fad01f4, Bolt auto-commit, fresh statement text, against the pinned neo4j:5.26.30-community:

statement Neo4j 5.26.30 NornicDB
RETURN 9007199254740993 = $v AS v, v = 9007199254740992.0 false true
MATCH (n:KC) WHERE n.id = $id RETURN n.name, stored id: 9007199254740993, id = 9007199254740992.0 no rows the node

Unlike the ordering operators in the table above, which Neo4j also compares as floats, Neo4j's = between an INTEGER and a FLOAT is exact. NornicDB converts the integer to a float, so the two values are equal.

Consequence

A lookup by a 64-bit ID sent as a float (JSON clients, dynamic languages) matches a different entity whose ID rounds to the same float. The result is silently wrong.

Expected

INTEGER = FLOAT is true only when the float is exactly that integer, as in Neo4j. Ordering between an INTEGER and a FLOAT stays as Neo4j does it.

🤖 Generated with Claude Code

Moved from #657: division and modulo by zero don't follow Neo4j's rule: runtime 1 / 0 returns Infinity, and % with a float operand fails (was #657 (comment))

Main 2afc84f against neo4j:5.26.30-community, Bolt auto-commit. Operands come in four ways: runtime (UNWIND [$a] AS a UNWIND [$b] AS b RETURN a <op> b), both parameters (RETURN $a <op> $b), all literals (RETURN 1.0 / 0), and a variable with a literal divisor (WITH $a AS a RETURN a <op> 0).

Neo4j 5.26.30's rule (measured):

  • /, integer 0 divisor → ArithmeticError ("/ by zero"), whatever the dividend's type: 1 / 0, 1.0 / 0, a / 0 with a = 1.0, n.f / 0. The one exception is an expression of literals only, RETURN 1.0 / 0, which Neo4j folds at compile time to Infinity. (RETURN 1 / 0 literal is still an error.)
  • /, float 0.0 divisor → IEEE: 1 / 0.0 and 1.0 / 0.0 give Infinity, -1.0 / 0.0 gives -Infinity, 0.0 / 0.0 gives NaN, in every form.
  • %: an error only when both operands are INTEGER (1 % 0). With any FLOAT operand the result is NaN: 1.0 % 0, 1 % 0.0, 1.0 % 0.0, 0.0 % 0.0, in every form.

NornicDB main differs in 26 of 48 cells:

form Neo4j 5.26.30 NornicDB main
runtime / param / variable int / 0 (e.g. UNWIND [1] AS a UNWIND [0] AS b RETURN a / b) ArithmeticError Infinity (silently)
runtime / param / variable float / 0 (integer zero) ArithmeticError Infinity
n.f / 0 on a stored float; WHERE n.f / 0 > 1 ArithmeticError Infinity; the row is kept
any form of float % 0, int % 0.0, float % 0.0 NaN ArithmeticError
literal 1.0 / 0, all / 0.0 forms, 1 % 0, literal 1 / 0 as above same as Neo4j

Consequence

Wrong results, silently. An integer division by a zero that comes from data or a parameter, such as an average over an empty group computed by hand (total / count), returns Infinity where Neo4j fails the statement. The Infinity is then stored, compared or returned. WHERE x / 0 > 1 keeps rows Neo4j would reject. In the other direction, modulo with a float operand fails statements that Neo4j runs.

The rule to implement: / fails only when the divisor is an INTEGER zero (a folded all-literal float / 0 gives Infinity); % fails only when both operands are INTEGER. Everything else is IEEE (±Infinity / NaN). The same rule applies on every evaluator path (context, row, compiled WHERE).

🤖 Generated with Claude Code

Moved from #657: invalid temporal text returns null (or is stored as its expression text) instead of failing (was #657 (comment))

Neo4j fails a temporal constructor whose string can't be parsed: SyntaxError "Text cannot be parsed to a DateTime / LocalTime / Time / LocalDateTime / Duration", and "Invalid value for MonthOfYear (valid values 1 - 12): 13" / "Invalid date 'FEBRUARY 30'" for out-of-range fields. NornicDB returns null for every constructor except date(), and date()'s error message differs. In a property map the expression is stored as its text.

Main 2afc84f against neo4j:5.26.30-community, Bolt auto-commit:

statement Neo4j 5.26.30 NornicDB main
RETURN datetime('x'), localtime('x'), time('x'), localdatetime('x'), duration('x') SyntaxError "Text cannot be parsed to a …" null
RETURN datetime('2020-02-30T10:00') SyntaxError "Invalid date 'FEBRUARY 30'" null
WITH 'x' AS s RETURN datetime(s); UNWIND ['x'] AS s RETURN duration(s) SyntaxError null
CREATE (n:TP {d: datetime('x')}) RETURN n.d SyntaxError, nothing created stores the string "datetime('x')"
MATCH (n:TP) WHERE datetime('x') IS NULL RETURN count(n) SyntaxError every row matches
RETURN date('x') / date('2020-13-01') SyntaxError "Text cannot be parsed to a Date" / "Invalid value for MonthOfYear …" SyntaxError "could not parse RETURN expression …" (same code, different message)
RETURN duration('P1Y') P1Y same

Consequence

Wrong data, silently. An import that converts date strings with datetime(row.ts) stores null for every malformed or impossible timestamp, and in a CREATE property map the literal text "datetime('x')", instead of failing on the bad row. The bad values are found only later, as missing or garbage dates. WHERE datetime(x) IS NULL matches every row with an unparsable value. On Neo4j the same statements fail at the first bad value.

The stored-text case is the #656 family ("property value stored as its expression text"). The fix should make the temporal constructors fail the statement on every route (context evaluator, row evaluator, CREATE property maps), through the one function-error path #698 is converging.

🤖 Generated with Claude Code

Moved from #657: an error inside CASE, a list or map literal, or coalesce() is swallowed and a wrong value is returned (was #657 (comment))

A runtime error in an expression nested inside CASE, a list or map literal, coalesce() or toString() doesn't fail the statement. The row gets a wrong value instead. Neo4j fails the statement. Main 2afc84f, with UNWIND [1] AS a UNWIND [0] AS b:

expression Neo4j 5.26.30 NornicDB
RETURN a / b ArithmeticError "/ by zero" ArithmeticError (correct)
RETURN CASE WHEN true THEN a / b END ArithmeticError 1
WITH CASE WHEN true THEN a / b END AS x RETURN x ArithmeticError 1
RETURN [a / b] ArithmeticError 1
RETURN {k: a / b} ArithmeticError 1
RETURN toString(a / b) ArithmeticError 1
RETURN coalesce(a / b, 2) / RETURN coalesce(1/0, 2) ArithmeticError 2
RETURN CASE WHEN true THEN 1/0 END ArithmeticError SyntaxError "could not parse"

Consequence

Wrong results, silently. A computation that fails inside a CASE, a collection, coalesce() or toString() returns an unrelated value instead of an error: here the dividend 1, or coalesce's fallback. Statements that should fail succeed and store or return that value. coalesce(x / y, default) hides every division error behind the default.

The fix is the same rule as the division one above: an evaluation error inside any nested expression fails the statement, on every evaluator path, with no fallback that replaces the failed sub-expression with a value. (#698 on #720 fixes only the toString(a / b) form.)

🤖 Generated with Claude Code

Moved from #657: temporal constructors from a map roll out-of-range fields over instead of failing (was #657 (comment))

An out-of-range field in a temporal map is normalized into the next or previous unit (Go time.Date behaviour) instead of failing. Neo4j 5.26.30 raises Neo.ClientError.Statement.ArgumentError. Main 2afc84f (also on #720 at f7813049):

statement Neo4j 5.26.30 NornicDB
RETURN date({year: 2020, month: 13}) ArgumentError 2021-01-01
RETURN date({year: 2020, month: 0}) ArgumentError 2019-12-01
RETURN date({year: 2020, month: 2, day: 30}) ArgumentError 2020-03-01
RETURN datetime({year: 2020, month: 1, day: 1, hour: 25}) ArgumentError 2020-01-02T01:00
RETURN localtime({hour: 24, minute: 61}) ArgumentError 01:01

Consequence

Wrong dates, silently. Data built from component fields (date({year: r.y, month: r.m, day: r.d}) in an import) turns an invalid date into a different real date, such as February 30 becoming March 1, instead of rejecting the row. The stored value looks valid, so the error is never noticed.

🤖 Generated with Claude Code

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions