fix: DynamoDB expression evaluation, UpdateExpression paths, and ConsumedCapacity

[diopolgg]

Summary

This PR fixes several DynamoDB compatibility issues and adds ConsumedCapacity support:

1. Replace regex-based expression evaluation with AST parser (fix)

The old regex/string-splitting approach for filter, condition, and key condition expressions broke on several valid DynamoDB patterns:

• <> comparisons on BOOL attributes (missing attribute handling)
• IN operator (fell through to pass-all)
• OR / NOT logical operators (only AND was implemented)
• Parenthesized BETWEEN: pk = :pk AND (sk BETWEEN :start AND :end)
• Compact format without spaces: (#f0 = :v0)AND(#f1 BETWEEN :v1 AND :v2)
• Numeric ExpressionAttributeNames (#0, #1) — relates to [BUG] DynamoDB: numeric ExpressionAttributeNames (#0, #1) not resolved — affects Query KeyConditionExpression and TransactWriteItems ConditionExpression #127

Adds ExpressionEvaluator with a proper tokenizer → recursive-descent parser → AST evaluator. Also fixes GSI sparse-index semantics: items where any GSI key attribute is null/missing are now excluded from GSI query results, matching real DynamoDB behavior.

2. Support dotted paths in UpdateExpression and fix clause boundary parsing (fix)

Two bugs in applyUpdateExpression:

• Dotted paths (e.g. #details.subkey = :val) were not resolved segment-by-segment — the entire string was looked up as a single ExpressionAttributeName, failing silently. Now each segment is resolved independently and nested Map nodes are navigated/created as needed.

• SET/REMOVE clause boundary — findNextComma was checked before findNextClauseKeyword, so SET #a = :v1 REMOVE #b,#c would treat the comma in the REMOVE list as a SET separator, corrupting both clauses.

3. Add ConsumedCapacity in DynamoDB responses (feat)

• Returns ConsumedCapacity in GetItem, PutItem, DeleteItem, UpdateItem, Query, Scan, BatchGetItem, BatchWriteItem when ReturnConsumedCapacity is TOTAL or INDEXES.
• For INDEXES level, includes per-table and per-GSI capacity breakdown.
• Adds ProvisionedThroughput to GSI descriptions in DescribeTable (was missing, causing clients that read GSI throughput to fail).
• Simple capacity estimates: 0.5 RCU per item read, 1.0 WCU per write.

Tests

Unit & integration tests (main project)

• ExpressionEvaluatorTest: 64 tests covering tokenizer, parser, key-condition splitting, and evaluator (logic, comparisons, functions, nested parens, compact format, expression attribute names)
• DynamoDbServiceTest: new tests for dotted path SET/REMOVE, SET+REMOVE clause boundary, compact BETWEEN queries
• DynamoDbIntegrationTest: 4 new integration tests for ConsumedCapacity (TOTAL, INDEXES, omission when not requested)
• DynamoDbFilterExpressionIntegrationTest: 14 new HTTP-level integration tests for BOOL <>/=, IN operator, OR/NOT operators, nested parentheses, parenthesized BETWEEN, and compact-format key conditions

All 1468 tests pass.

Compatibility tests (SDK against live Floci)

• DynamoDbExpressionTests (new, 14 tests): end-to-end AWS SDK v2 tests covering filter expressions (BOOL, IN, OR, NOT, nested parens), dotted UpdateExpression paths, ConsumedCapacity (TOTAL/NONE for Scan/GetItem/PutItem), and parenthesized BETWEEN in KeyConditionExpression (standard and compact format)
• Surefire include fix: added **/*Tests.java pattern to compatibility-tests/sdk-test-java/pom.xml — several existing test classes (DynamoDbScanConditionTests, EcsTests, Ec2Tests, etc.) were silently skipped because surefire only matched **/*Test.java

All 38 DynamoDB compatibility tests pass (18 existing + 6 previously skipped + 14 new).

[hectorvent]

Hi @diopolgg,

Thanks for taking the time to contribute! We've seen your PR and will take a look soon.

To ensure this implementation stays robust, could you also open a PR in our compatibility suite at https://github.com/hectorvent/floci-compatibility-tests? This helps us validate that your changes keep working perfectly as the project evolves.

Also check conflicts with main branch,

The Floci team

[hectorvent]

Hi @diopolgg, tests project embedded directly in this project please sync with main, solve conflicts and get the test implemented. Test directory is compatibility-tests.

Thanks

[hectorvent]

Conflicts changes required

[diopolgg]

Resolved merge conflicts with latest main. All unit tests (1725) pass, and Java compatibility tests (DynamoDB expression, scan condition, and all other services) pass against a live Floci instance. Ready for re-review.

[hectorvent]

Hello @diopolgg,

Thanks for the PR!

Could you please update the commit history to remove the Co-authored-by: Claude <...> line from the commit message? We prefer to keep the attribution focused on human contributors for this project. Once that's cleaned up, I'm happy to take another look!

[diopolgg]

@hectorvent Rebased again, removed claude code attributions.. seems conflicts appear pretty quickly here :). Sometimes quicker then we can make yet another review roundtrip. Thus hard to squash/merge. Hope it will be merged eventually though. We have large codebase with integration tests for DDB. Most of the fixes i did were just to make floci work well for us. With this changes it works fine. Great project btw! Big thanks for it.

[hectorvent]

Hi @diopolgg,

Thank you for your patient and for keeping this PR aligned with latest changes on main.

I see some tests errors, but they are not related to your PR. I will merged it and resolve those issues in another PR.

Thank you again!